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I INTRODUCTION, 

The paper which follows is based on notes taken by R, S. Pierce on 
five lectures given, by the author at the California Institute of Technology in 
January 1952 „ They have been revised by the author,, but they reflects apart 

from stilistical changes, the lectures as they were delivered. The author 
intends to prepare an expanded version for publication, and the present write 
up, which is imperfect in various ways, does therefore not represent the 
final and complete publication. That will be more detailed in several respects, 
primarily in the mathematical discussions of sections 9 and 10 (especially in 
10. 2 and 10. 5. 2), and also in some of the parts dealing with logics and with 
the synthesis of automata. The neurological connections may then also be 
explored somewhat further. The present write up is nevertheless presented 
in this form, because the field is in a state of rapid flux, and therefore for 
ideas that bear on it an exposition without too much delay seems desirable. 

The analytical table of contents that follows will give a reasonably 
close orientation about the contents - indeed the title should be fairly self- 
explanatory, The subject-matter is the role of error in logics, or in the 
physical implementation of logics - in automata - synthe sis . Error is viewed, 
therefore, not as an extraneous and misdirected or misdirecting accident, 
but as an essential part of the process under consideration - its importance 
in the synthesis of automata being fully comparable to that one of the f&feior' 
which is normally considered, the intended and correct logical structure! ’ 

Our present treatment of error is unsatisfactory and ad hoc. It.’, 
is the author’s conviction, voiced oyer many years, that error should .be; , ' 

treated by thermodynamical methods, and be the subject'of a thermodynamical 
theory, as information has been, by the work of L. Szilard and O. E. Shannon, 
(Of. 5,2) The present treatment falls far short of achieving this,'but it 
assembles, it is hoped, some of the building materials, which will have to' 
enter into the’final structure, ' ' ’’)■ 

The author wants to express ids thanks to K. A. Bxueckherhind , 

M. Gell-Mann, then at the University of Illinois, to discussions with whom 
in 1951 he owes some important stimuli on this subject! to R. S. Pierce at, 
the California Institute of Technology, on whose excellent notes this ex¬ 
position is based; and to the California Institute of Technology, whose in¬ 
vitation to deliver these lectures combined with the very warm reception 
by the audience, caused him to write this paper in its present form, 

2. A SCHEMATIC VIEW OF AUTOMATA. 

2, 1 Logics and Automata. 

It has been pointed out by A. M. Turing (5), in 1937 and by 
W. S, McCulloch and W. Pitts (2) in 1943 that effectively constructive 
logics, that is, intuitionistic logics, can be best studied in terms of 
automata. Thus logical propositions can be represented as electrical net¬ 
works or (idealized) nervous systems, ¥fhereas logical propositions are 
built up by combining certain primitive symbols, networks are formed by 
connecting basic components, such as relays in electrical circuits and 
neurons in the nervous system. A logical proposition is then represented 
as a "black box" which has a finite number of inputs (wires or nerve 
bundles) and a finite number of outputs. The operation performed by the 
box is determined by the rules defining which inputs, when stimulated. 
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cause responses in what outputs, just as a propositional function is determined 

by its values for all possible assignments of values to its variables. 

There is one important difference between ordinary logic and the automata 
which represent it. Time never occurs in logic, but every network or nervous 
system has a definite time Lag between the input signal and the output response. 

A definite temporal sequence is always inherent in the operation of such a real 
system. This is not entirely a disadvantage. For example, it prevents the 
occurence of various kinds of more or less overt vicious circles (related to 
” non "constractivity", "impredicativity", and the like) which represent a major 
class of dangers in modern logical systems. It should be emphasized again, 
however, that the representative automaton contains more than the content of 
the logical proposition which it symbolizes - to be precise, it embodies a 
definite time lag. 

Before proceeding to a detailed study of a specific model of logic, it 
is necessary to add a word about notation. The terminology used in the foiiow- 
ing is taken from several fields of science; neurology, electrical engineering, 
and mathematics furnish most of the words. No attempt is made to be system¬ 
atic in the application of terms, but it is hoped that the meaning will be clear 
m every case. It must be kepi in mind that few of the terms are being used 
in the technical sense which is given to them in their own scientific field. 

Thus, in speaking of a neuron, we don't mean the animal organ, but rather 
one of the basic components of pur netwqrk which resembles an animal 
neuron only superficially, and which might equally well have been called an 
electrical relay. 

2. 2 Definitions of the fundamental concepts. ' 

Externally an automaton is a "black box" with a finite number of ' 
imputs and a finite number of outputs. Each'input and each output is capable 

of exactly two states, to be designated as the "stimulated" state and the 
"unstimulated" stats, respectively. The internal functioning of such a 
"black box’* is equivalent to a prescription, which specifies what outputs 
will be stimulated in response to the stimulation of any given combination of 
the inputs,and also of the time of stimulation of these outputs. As stated 
above, it is definitely assumed that the response occurs only after a time 
lag, but in the general case the complete response may consist of a succes¬ 
sion of responses occurring at different times. This description is some¬ 
what vague. To make it more precise it will be convenient to consider first 
automata of a somewhat restricted type and to discuss the synthesis of the 
general automaton later. 

DEFINITION 1: A single output automaton with time delay 6 (A -is 

positive) is a finite set of inputs, exactly one output, and an enumeration 
of certain "preferred" subsets of the set of all inputs. The automaton stim¬ 
ulates its output at time t + q if and only if at time i the stimulated inputs 
constitute a subset which appears in the list of "preferred" subsets, 
describing the automaton. In the above definition the expression "enumera¬ 
tion of certain subsets" is taken in its widest sense and does not exclude 
the extreme cases ’ all" and "none". If fl is the number of inputs, then 
there exist % va ' such automata for any given 6 . 

Frequently several automata of this type will have to be considered 
simultaneously. They need not all nave the same time delay, but it will be 
assumed that all their time lags are integral multiples of a common value 
. This assumption may not be correct for an actual nervous 
system, the model considered may apply only to an idealized nervous system. 

In partial justification, it can be remarked that as long as only a finite number 
ot automata are considered, the assumption of a common value can 

oe re ^ 12ed wi . thin any degree of approximation. Whatever its justification 
and whatever its meaning in relation to actual machines or nervous systems. 



this assumption will be made in our present discussions. The common value 

is chosen for convenience as the time unit. The time variable can now 
be made discrete, i. e, it need assume only integral numbers as values, and 

correspondingly the time delays of the automata considered are positive 
inte gers. 

Single output automata with 'given time delays can be combined into a 
new automaton. The outputs of certain automata are connected by lines or 
'vires or nerve fibers to some of the inputs of the same or other a.utomata, 
'The connecting lines are used only to indicate the desired connections; their 
function is to transmit the stimulation of an output inatanteously to all the 
inputs connected with that output. The network is subjected to one condition, 
however. Although the same output may be connected to ‘several inputs, any 
one input is assumed to be connected to at most one output. It may be clearer 
to impose this ‘restriction on the connecting lines, by requiring that each in¬ 
put and each output be attached to exactly one line r to allow lines to be split 
into several lines, but prohibit the merging of twq or more lines. This con- 




an input, and hence of'a line, is an all or nothing process. If a line is 

split, the stimulation is carried to all the cxanches^in fuU. No energy con¬ 
servation Taws enter into the problem. .In actual machines or neurons,' the 

,energy is' Shpplftft .by ’ the neurons /themselves Cram some'external source 
• of energy, £"The' stimulation acti'^oaly as trigger' devied.' - ;' ‘ \ 

The" most general automaton is defined to be any such network. In' . 
gener aT it will hS.ve several inputif'and"several outputs aud its response ‘ • 
activity will be. much mqre complex than that ©£ a single output automaton 
with a given time delay, ;; :.i4&'£ntrinsic .definition.of the. general automaton,- 
independent of. its cpasbruetiba ab-a networkilean be supplied. It/will‘not 
be discussed here, however. 


Of equal importance to the problem of combining automata into new 
ones is the. converse problem of. representing a given automaton by a net¬ 
work of s imple^'aufcOma^'; and b£ determining eventuality a'minimum . 

number of basic type s 1 for' thfese simpler automata / 1 As will be shown, very 

few types are necessary.' ’ -! . •' ■ • • ■ 


2. 3 Some basic organs. 

■ . i . . .v. 


The automata to be selected as basis for the synthesis of all 

automata will be called basic organs. Throughout what follows, these 
will fee single output automata. 

One type of basic organ is described by Figure l. It has one out¬ 
put, and may have any finite number of inputs. These are grouped into 

two types; Excitatory and inhibitory inputs, ’ The excitatory, inputs are 
distinguished fro'm the inhibitory inputs by the addition, of an arrowhead 

to the former and of a small circle to the latter. This distinction of in¬ 
puts -into two types does actually not relate to the concept of inputs, it is 
introduced as'a means to describe the internal mechanism v of the neuron. 
This. mechanism is fully described fey the so-called threshold function 
P W written inside the large circle symbolizing the neuron in Figure 1, 
according to the following convention: The output of the neuron is excited 
at iimejfr*/ if and only if at time t the number of stimulated excitatory 
inputs A and the number of stimulated inhibitory inputs jt satisfy the 
relation H k <p(&. (It is reasonable to require that the function ^?(x) be 
monotone non-decreasing.) For the purposes of our discussion of this 
subject it suffices to use only certain special classes of threshold func¬ 
tions PCX) . E„ g P 
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(i. e. < ft inhibitions are 
effective), or 


absolutely ineffective, j> ft: inhibitions are absolutely 


(2) <P CX) == Xf t (*) = X+ fl 

(i, e, the excess of stimulations over inhibitions must be k /q ). We will use 
Xa , and write the inhibition number % (instead of %£ ) inside the . 

cirule symbolizing the neuron. Special cases of this type are the three basic 

organs shown in Figure 2. These are, respectively, a threshold two neuron 
with two excitatory inputs, a threshold one neuron with two excitatory inputs, 
and finally a threshold one neuron with one excitatory input and one inhibitory 

input. ' t 

The automata with one output and one input described by the networks 

shown in Figure 3 have simple properties: The first one’s output is. never ^ 
stimulated, the second one’s putput is stimulated at all times if its input has 

been ever (previously) stimulated. Rather than add these automata to a net¬ 
work, we shall permit lines leading to an input to be either always non-stimu¬ 
lated, or always stimulated. We call the latter “grounded" and designate it 

by the symbol \\ I- and we call the former "live” and designate..ft by 

the.symbol —— . ... ... . .... . ... . •, 

3. AUTOMATA AND THE PROPOSITIONAL CALCULUS. 

T; -fj ; . 

3.1 The propositional calculus. 

The propositional calculus deals with propositions irrespective of .' 
their truth. The set of propositions is closed under the,operations of 
negation, conjunction and disjunction. If CS. is a proposition, .then "not ...y* 

CL ", denoted by ctT* (we prefer this designation to the more conventional 
ones - a and ^O,), is also a proposition. If GL.Cr are two proposi¬ 
tions, then" a and & ", " a or (r ", denoted -respectively by Ctfr 
are also propositions. Propositions fall into two-sets, 7" and F , depenn- 
ing whether they are true or false. The proposition CL is in 7 if and 
only if a is in F . The proposition Q& is in T if and only if a and tr 
are both in T » and is in T if and only if either <2 or tr is in V . 
Mathematically speaking the set of propositions, closed under the three 
fundamental ^operations, is mapped by a homomorphism onto the Hooiean 
algebra of the two elements J_ and O . A proposition is true if and only ,,. 
if it is mapped onto the element | , For convenience; :denote by J, ■ - 

the proposition <14 ( fay 0 the proposition &. Q. ‘ .•-Whcye tk fixed 

but otherwise arbitrary proposition. Of cour se, $ is false,: and £ is true. . 

A polynomial P in n variables, ntt , is any formal expression 
obtained from k, } ...» by applying the fundamental operations to them 

a finite .number of times, for example [( X 4 4* Xg ! ) 1 * ' is a poly¬ 
nomial. In the propositional calculus two. polynomials in the same vari¬ 
ables are considered equal if and only if for any choice of the propositions 
X|j ^ Xr t the resulting two propositions are always either both true or 
both false, b A fundamental theorem of the propositionalfcalcuius states that, 
every polynomial 3^ is equal to 


Z ... I- fr.L X, 1 ".,, *:* 
£ s ±\ ' n 


! 


n p 
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where each of the fl ,... i n is equal to 5 or ! . Two polynomials are equal 
if and only if their f *s are equal. In. particular, for each. n, , there exists 

exactly polynomials. 

3.2 propositions; automata and delays. 

These remarks enable us to describe the relationship between auto¬ 
mata and the propositional calculus. Given a time delay 3 , there exists a 

one-to-one correspondence between single output automata with time delay 
S and the polynomials of the propositional calculus. The number ft of in¬ 
puts (to be designated l s ... , rv } is equal to the number of variables. Fox 
every combination 1 3 — tl « the coefficient fl, ... L n * J. i 

and only if a stimulation at time t oi exactly those inputs tJ for which h 
produces a stimulation of the output at time t$ £ . 

DEFINITION 2: Given a polynomial at $ Xn) and a time delay S , 

v/e mean by a -network a network built from the three basic organs o£ 

Figure 2 e which as an automaton represents ip with time delay ft . 

THEOREM 1; Given any f* , there exists a (unique) 6*s6 (f) , such that a 

«jPg -network exists if and only if £<£ ft*. 

PROOF: Consider a given J 3 * . - Let be the set of those & for which a 

-network exists. If ft'iS , then tying tf-ft unit-delays, as shown in 

Figure 4, in series to the output of & ijt -network produces a -net¬ 
work,, Hence '&CP) contains 1 with an» all safe's * . Hence if is not 

empty, then it i’i ..precisely the set of all &%£>*, where ft*® s&CPJis its 

smallest element. Thus the theorem holds for P if SCPJ is not empty, i. e. 
if the existence of at least one -network (for some* 1) is established. 



Fig. 2 




Fig. 4 
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Now the proof can be effected by induction over the number p — (S^) 

of symbols used in the definitory expression for # (counting each occur- 
rence of each symbol separately). 

If « S , then >»« 4 *n )s Ky (for one of the a ), 

The ''trivial'* network which obtains by breaking off all input lines other than 
y , and taking the input line V directly tc the outputs solves the problem 

with s««0 . Hence §*(?}■= 0 . 

If p (¥) > I s then (2 or #= (2^? or . where pf<2) 

pC^)<p(^. For (fs fit"* let the box [(STl represent a (oL j £>' - network, with 

5 ~ £>*C(2.) . Then the network shown in Figure 5 is clearly a -network, 

with 5 = sVl . Hence S* C<?) ^ S*hS}-t-t, For 5% Q^.or let the boxes 

represent a (•2, s' 7 -network and an & 3 &' -network, respectively, 

with =: Moot (S ff CSn j £?£&)}♦ Then the network shown in Figure 6 is clearly 

a iPjfe -network, with <2i? or-Sfi? for & Q, or i , respectively, and 

with S s I * Hence. Msjj (s *(&) t -4- I 

Combine .the above theorem with the fact, that every single output 1 
automaton can be equivalently'described --.apart from its time delays —" 
by a polynomial.# 8 , and'that the basic operation dfc , Q.+ fir , QT' of the _ 

propositional calculus are represented (with unit delav) by the basic organs 

of Figure 2. (For the last one, which represents a 6-“*, cf. the remark at 

the beginning of 4. 1. 1.) This gives; 

DEFINITION 3: Two single output automata are equivalent in the wider 
sense, if they differ only in their time delays -- but otherwise the same 

input stimuli produce the same output stimulus (or non-stimulus) in both. 

THEOREM 2 (Reduction Theorem): 


(< 2 .! , JR, 


Any single output automaton Ol is equivalent in the wider sense to 
a network of basic organs of Figure 2. There exists a (unique) &*^ss S* COl) , 
,such that the latter network exists if and only if its prescribed time delay s 

satisfies vS =?s* . 







3, 3 Universality. General logical considerations. 
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Now networks of arbitrary single output automata can be replaced by 

networks of basic organs of Figure 2; It suffices to replace the unit delay in 
the former system by § unit delays in the latter, where S is the maximum 
ot The s (in.) of all the single output automata that occur in the former 
system. Then all delays that will have to be matched will be multiples of 
5 , hence ^ s ^ hence. | S COL ) far all that can occur in this situa- 

tioti, and so tlie Reduction Theorem will be applicable throughout. 

. Thus tMs system of.basic organs is universal: It permits the con¬ 
struction, of'essentially equivalent networks to any network that can fee con¬ 
structed from any system o£ single output automata. I. e. Kbl redefinition of 
the system of basic organs can extend the logical domain covered by the de¬ 
rived networks. 1 


lhe general automaton is any network of single output^iutevnata is 
the above sense. ' It must be emphasised, that, in particular, feedbacks, i. a. 

arrangements of lines which may allow cyclical stimulation sequences, are * 

allowed. (I. e. configurations like those shown in Figure 7, etc. There will 
be various, non-tri%aal, examples of this later,} The above arguments have 
shown, that a .limitation of the underlying single output automata to eu'r " • 

onpnal basic-organs causes no essential loss of generality. The 1 Question. 

as to which logical operations can be equivalently represented' (with suitable *1 

but not specified, , delays) is nevertheless not without difficulties. i;:j * 

These general automata •sn .* 1 A : <8 ^ JSJt JL'22 V ' _• • >• •* 




-very constructive Imtuitiouistic} logics. ‘-Ite. i^aiveifa Problem 
involving (a'finite number of) variables, which can be solved (idihiic&Uy in^ 

ve construction; it is not alwa vs v pb'sb ible'lto" can'i- 


,» * TTMa%«u. uau WIB »u&vca*| iaenUCftiLV % 

these variables) by elective construction, it is not aiways'pdshibleHo* con 
struct a ffeneral ; «« 


-- tiaiiy# was me memorv 

«f^ ire ‘ m w ntS;8UC ? a P robIem ma 7 depend on (actual values ashuinhd b^" 
the variables (i.e. they must be finite for any specific. 1 system' 5 of values of 
the variables,. but they may be unbounded for the totality of all possible 
systems of values), while a general automaton in the above sense neeessar- 
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4 BASIC ORGANS, 


4. 1 Reduction of the basic components. 

4. 1. 1 The simplest reductions. 

The previous section makes clear the way in which the eiem.~nta.rv 
neurons should be interpreted logically. Thus the ones show! fa 2 

re pectively represent the logical functions «fr, , and af?~ f T n 

afshotnTn Flsrure^S s ^' £ices ^ mak f the 0 -terminal of the third organ, 

sLv^Tn Sgurfg. 6 8 * UVe ' ThlS WUl be abbrCViated in the as 

_ Now since a& s (ccT 1 ) * and a + fr & ((Cl*) ( 

Fiflruri e z i tKat * he , flrS !: ° rgan amon ^ the three basic organs shown in ’ 

e <^valen S to a system built of the remainini two organs there 

Sf h r T d n ?K an abbreviation for a composite, as in Figured), an/Sther'" 
the first or the second basic organ in Figure 2, are the basic organs '-They 

permit forming the second or the first basic organ in Figure 2, respective-* 
ft* a " Sb ° Wn above ’ as (composite) networks. The third basic organ in 

f easily seen to be also equivalent (in the wider sense) to a com¬ 
posite of the above, - but, as was observed at the beginning of 4 l l tbf 
necessary organ is in any case not this, but —^l g (cf alsoV • 
remarks concerning Figure 8), respectively. Thus either system of new 
owarts° PC ™ itS i reconstructing (as composite networks) all the (basic) 

v g . - nginal system. It is true, that these constructs have delays 

fc ° 3f but Since unit de . la y s » as shown in Figure 4 are avail. 

3 " Th l J: 1 £?. w s >stem 5 all these delays can be brought up to the value 

3. Then a trebling of the unit delay time obliterates all differences 

a-ain in Fi^r^TO InStead f th f three ori g inal basic organs, show 
a a am m l igure 10, we can also (essentially equivalently) use the two basic 
organs Nos. one and three or Nos.*two and three in Figure 10 

4. 1. 2 The double line trick. 

combination «“»' conaio.c the one remaining 

ccmnmation, too. The two basic organs Nos. one and two in Figure 10 
as the basis of.an essentially equivalent system. 8 ’ 

* ° n f r °.^ ld be iTlclined infer that the answer must be negative- 
eauivaUnM-i^rt ° U -/ t th&: fcw ® first basic organs of Figure 10 cal be ' 

g “ - d ’ the S! - a “ e OI the output is a monotone non-decreasing 


Fig. 8 


i<u 


Xl > 


a (D~ 


Fig, 9 

Fig. l'O 


=g^D-o- 

urs®— =£T>^ —<0— 
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1 the n in P" ts both basic organs Nos. one and two in 
Figure 10, and hence for all networks built, from these organs exclusively as 

T n L~Jr 13 ’ °Tt Ver: iS " 0t the case for the last organ of Figure 10 (norfor 
ale last organ of Figure 4 ), irrespectively of delays. ^ 

Nevertheless a slight change of the underlying definitions permits one 

F°™rr^r?~ d Th^e'v IO<i ?.*! t 2" " »* <* h « T 

interest bee" ust U rZ 'Z ZZ eUects this is of additional methodical 

UteJ on ina mot onnl^tT-h the prototype of one that we will use 

a . on in a more tompneated situation. The trick in Question is tm 

r £?to!' S TaU Une ! nSt ' ad o£ 3in8le one - °"' •“»» that if 

me two lines, at all times precisely one is stimulated. Thus there will 
always be two possible states o£ the line pair: The first line stimulated 

Original SV J™ ° f ZZ :t t eS C ° rres P^ d fcbe stimulated single line of the 

JLi it e toT p f °,t io p n r ™o“ rt: ,T at f- td ihe 

ZZnEttfFT ISZTriXi 

original system of Figure 2 ,} relates to the 

2 un.timuUtld w* leTfcl .V™ pr °f. 0,itiOT corresponds to 1 atimuUted, 

har. .t.o the co^ecTdelaya r?i“ Lut ° f “ lird 

whenever thi. i. not wanted, It oan be'ne'plac^ nnlTdeU? *S°rSl«ino"‘ 

-- sraSrfc&^i ?wj2sk i° 

Sg^roSr 3 - aad yield * sysiem tot. 0 be 

4, 2 Single basic organs. 

'*• 2. 1 The Schaffer stroke. 

though it cannot be done^with any* of /he thr^ 6 ** ° f ° asic or S ans to one » al- 
will, however, introducer, ZL three organs enumerated above. We 

The organs, either of which suffices by itself. 

stroke” function l£ u .*in thU % ° toe w.Il-known»'Sche«er 

and G. Gell-Mann * TnlZh^t 5 :f ntext , was suggested by-K. A.- Brueckher 
shown on figure 1*2 ~ Thi «,«*% %t be V epresented (and abbreviated) as 

performed in Fi^ire 13^ ^ now' be 

cation .1“* *iS U «“ e «”P«- 

put of the Scheffer .tr-iite “ s ag se»tisl s Indeed,, the ouf- 

in everTnetorrdVrrvtl from xV ^eTdSK of if s inputsl Hence 

functions of its immts Zi ?*, ^ e ) ay ? ut ? u£s ^ be m^uotanefc,,*,- 
How ot &'and Je r^ <mtputs antimonotone onesr. 

Hence no delay-vaT^ “* are fi0t monotone, 

two first erg... and one ofX’ Ir^nT'**** *” tMs *“ “ p ™ <* «* 

i h ni":, b “f: rd ° m % a ' ,ollowS! «*— 

our earlier result fin 4 1 zV «eru^?nJ fi, , ame delay, namely 2, Hence 
two basic or»an= a& and „!* g ad ®9L uac >' ° 4 4fc -e system of the 

g GC ^ anG arCr a P? hes * Doubling the unit delay time re- 
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duces the present set up (Scheffer stroke only!) to the one referred to above, 
4, 2. 2 The majority organ. 

The second universal organ is the majority organ". In symbols, it 

is shown (and alternatively designated) in Figure 14. To get conjunction and 

disjunction, is a^simple matter, as shown in Figure 15. Both delays are-1. 
Thus Clfjr and Q.&& (according to Figure 10) are correctly represented, and 
the new system (majority organ only!) is adequate because the system based 

on those two organs is known to be adequate (cf. 4. 1. 2). 

Fig. 11 


a(y — 


organ No. 


1 of Figure 2 or 10: 



• t 

CL - organ No. 3 of Figure 10: 




(all-) i tafj) ' : 


Figure 12. 



Figure 13* 









5 IDG1CS AND INFORMATION. 


5. 1 Intuition is tic logics. 

All of the examples which have been described in the last two sections 
have had a certain property in common: in each, a stimulus of one o£ the in¬ 
puts at the left could be traced through the machine until at a certain time 
later it came out as a stimulus of the output on the right. To be specific, no 
pulse could ever return to a neuron through which it had once passed, A 
system with this property is called circle'-free'by W. S, McCulloch and 
W. Pitts (2), While the theory of circle-free machines is attractive because 
of its simplicity, it is not hard to see that these machines are very limited 
in their scope. 

When the assumption of no circles in the network is dropped, the 

situation is radically altered. In this far more complicated case, the out¬ 
put of the machine at any time may depend on the state of the inputs in the 
indefinitely remote, past. For example, the simplest kind of cyclic circuit, 

as shown in FTgure id, is a kind of memory machine.' 'Once this organ h$s 

been stimulated by CL , it remains stimulated and sends forth a pulse in 6" 
at all times thereafter. With more complicated networks, we can construct 
machines which will count, which, will do simple arithmetic, and which will 

will even perform certain unlimited inductive processes. Some of these 
will be illustrated by examples in 6. The use of cycles or feedback in auto¬ 
mata extends the logic of cons true table machines to a large portion of in- 
tuitlonlatic logic. Not all of intuition!*tic logic is so obtained, however, 
since these machines are limited by their fixed size. (For this, and for 
the remainder of this chapters ef, also the remarks at the end‘6f 3.’3) ;i Yet, 
if our automata are furnished with an unlimited.memory «« for example, 
an infinite tapis, : arid.scanners connected to-afferent organs, • along with 1 
suitable efferent organs to perform motor operations and/or print on the 
tape -- the logic of constructable machines becomes precisely equivalent 
to intuition!stic logic (see A,M, Turing (5)). In particular, all numbers 
computable in the sense of Turing can be computed by some such network, 

5. 2 Information., 

5,2.1 General, observations. 

Our considerations deal with varying situations, each of which con¬ 
tains a certain amount of information. It is desirable to have a means of 

measuring that amount. In most cases of importance, this is possible. 
Suppose an event is one selected from a finite set of possible events. Then 
the number of possible events can be regarded as a measure of the informs - 
'tion content of knowing which event occurred, provided all events are a 
piuon equally probable. However, instead of using the number t% of pos¬ 
sible events as the measure of information, it is advantageous to use a 
certain nineties of fl , namely the Logarithm. This step can be ^heurisii— 
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caliy} justified as follows; If two physical systems I and II represent n an rt 

m <* equ.Uy prob.bl.) al.ernitives, respectively, Ih./S™ j«I re- 

present, not such alternatives. Now it ia desir ^ [e th J' the (numerical „el 
sure of information be fmimerieaiivl me a 

rfitive “*'™ call U adGlti « ^-der this (substantively) ad- 

o£ n , ''suet Sit CS 30lTie IUnctlon f <"> should be used instead 


f C n vn,) 


■f Cn 0 + f 6rt) 


sonaWe^to’require^ 1 repreSents mor£ ^formation that II, hence it is rea- 

( 4 ) n> tn implies f Cn) > fCYn). 

Note, that fCn) is defined for n. e i, 2. 

eludes easily, that ’ * ‘ 


only. From ^o)» (4) one eon- 


(5) fen 

for some constant C > 


.ft. 


13 ) alone does ' Since is defined for fix U,,„ only, 

10 / -ione does not imply tins, even not with a constant C t 6 ! Y Next it 

conventiona! to let the minimum non-vanishing amount oflnfoLS i l 

iim. means that rf«« I ■, i. s. Css »/&, 1 , and so 


^ ( 6 ) •£ Crj V s n#, - 

tatLTte 1 WO-T aldlkifl d r SlOPe ? I 7 "*•**“ «*»• 

system by c. E, 'shaunoh (3); *’ <ta *“’ r int0 * 

5. 2,2 Examples, * , 

of the flip "of a ebtafs g^STS “u T&JI* “1°?' 

M - ’' Sa a ®m K £.e character from a 44-key, 2- 


■ »*-i»b«sr a. . u a... w , 

nerve fibre which cap be classified as either sMmil.teH ** “7 T or 

carries precisely one bit of information, while I buidle of n'^cTr" 21 ^"' 1 ' 
communicate n. bits rt is ' 1 ~ , 016 ot n ‘ such lines can 

Sible only on * I- 0 *" 

namely, the knowledge of a system of a e « M In P \ ^v/leuge exists, 

This definite v / " P r OTl equally probable events. 

events are not iu e^Uy prob^ Suntose tJ ? here P OSsibie 

probabilities p, ‘a P e * " u PP° 5e tiie events are known to have 

ledge of which^of these**vents Then ,f ie mformaticm contained in the know- 
s vnitii ot these events actually occurs, is defined to be 

( i) Hs» t&g p. (bits). 

viou^orp * Thfe"' = ^7 *. definition is the same as 

if Z r This resuk, too, was obtained by C. E. Shannon 
• implicit m the earlier work of L. Szilard ( 4 ) ~ v 3> 

resemblance'To 5 th^stati^tical^d^^*’ 1 ^ thi /i efin ^ion is that it bears close 

system. If the possible * v -rts are^ius? i?,.? 6 entro Py ° { a thermodynamical 

p - -v-nta are just the known possible states of the 


xe pre- 
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system with their corresponding probabilities, then the two definitions are 
identical.. Pursuing this, one can construct a mathematical theory of the 
communication of information patterned after statistical mechanics. (See 
L*. Szilard (4) and C. E, Shannon (3),} That information theory should thus 
reveal itself as an essentially thermodynamical discipline, is not at all sur¬ 
prising; The closeness and the nature of the connection between informa¬ 
tion and entropy is inherent in JL, Boltzmanns classical definition of entropy 
(apart from a constant, dimensional factor) as the Logarithm of the "con¬ 
figuration number." The "configuration number" is the number of a priori 
equally probable states that are compatible with the macroscopic descrip¬ 
tion of the state -« i. e, it corresponds to the amount of (raiscroscopic) in** 
formation that is missing in the macroscopic) description. 

6 TYPICAL. SYNTHESES OF AUTOMATA. 

6 , 1 The memory unit. 

One of the beat ways to become familiar with the ideas which have 

been introduced, is to study some concrete examples of simple networks. 
This section is devoted to a consideration of a few of them. 

The first example will be constructed with the help of the three 
basic organs of Figure 10. It is shown in Figure 18. It is a slight r.e— 
fsntfcment of the primitive memory network of Figure 16. 

This network has two inputs &, and & and 'one output X * At 
time t , x is stimulated if and only is & has been stimulated at a a 
earlier time, so that no stimulation ai has occurred since then. Rough*. 
Iv speaking, the machine remembers whether gtiov was the last input -V 
to be stimulated. Thus X is stimulated, if it has been stimulated immed¬ 
iately before — to be designated by or if a has been.stimulated 

immediately before, but has not been stimulated immediately before. 

This is expressed by the formula X ■ C*'+*Q.) &•“*■ , i. e. by the network 
shown in Figure 1?. Now X should be fed back into X * (since xf is 
the immediately preceding state, of x ), This gives the network shown 
in Figure 18, where this branch of % is designated by % , However, the 
delay of the first network is 2, hence the second network's memory ex¬ 
tends over past events that lie an even number of time (delay) units back, 

L e. the output X is stimulated if and only if a has been stimulated at an 
earlier time, an even number of units before, so that no stimulation of 6* 
has occurred since then, also an even number of units before. Enumerat¬ 
ing the time units by an integer t . it is thus seen, that this network re¬ 
presents a separate memory for even and for odd t . For each case it 
is a simple "off-on", i, e. one bit, memory. Thus it is in its entirety a 
two bit memory, 

6 , 2 Scalers. 

In the examples that follow, free use will be made of the general 
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family of basic organs considered in 2. 3, at least for all f s 

fe'prb by the e Eldu°”o^eo to '"*»»• “ lhe Migfcil Ln« i., 

ed i n Ver B ret;«onr-c T " sub “«“» tl J' daveiop- 

It will be eall.dT'l«"? by L a o” iC £"ir hiCh '. C “? ’T 1 * stimali * two'. 

be burned off i\ n witr 0t ^be^ pUt ' the , ] ^pressor, the above mechanism can 

be .ttma oH afc will. The aiagram becomes as shown in Fitmre 20 The 

dicatsd^by Figure'20 * ”*~ U ' ^ ^ "«> • Jt 

be hocked t* ^ «» 

use of the raprassor is of course £tioLl here bX" h Th ' 

diffiLltv "£?*“'% ° f th * l ° m ^ ‘ can also >» constructed Vth IHtlV 

difficulty, but we will not go into this here, 

6. 3 .Learning. 

Using these "scalers hv c 71 >> « ~ \ . 

zzsrrsft “ g cs? 

then, in the next'instant, JXfu III^TTZh 

issarilv _occurs “®ss 



Utai'a hUbe« ««™"U°. a "‘ 1Cip '“ e a puise from » «>«««“ time* 

output after ««ry“Smilaa^ a*"™ 3- by bei " 8 st ?“' llated -« its *• 

& SS fea 

?!?•„ r.Z 3 :r "™? o °.i“ e ”«sirL m »idh"e f so'trLst 

64 C cotfnte^lexampl^s l tiT" n ^fo^ow^ b " n ff no n (posiu rn ) by •* 

r:“: rn Th"‘di ! y , .i‘i% (n - esativ ' ) o«»^M rs, ?, *“ s be . 

ive one has to use x tc gate at Jto repl^ffrlt^ i*"" 1 ?®, ? 

Let these be represented L - replace y di *ss normal functions. 

by the network shown in Figu~= 24^ TH«‘ " hen , tals ?7 ° ce55 is mediated 
the lines a, & Indto L if* !’ ™ ls netwc , rk then be attached to 

Figure 22,23), *’**'* ° ‘“ e preceding network {according to 

? THE ROLE OF ERROR, 

?* 1 _ Exemplification with the help cf the memory unit. 

components 1 * haS beeti that the basic 

ly not a very realistic one Jf l perf ? °5 mance « This assumption is clear- 
are statistically subpct'to failure^and J? eV1Ces * 8 WeU as ele c<=rical ones 

neurons too. Hence it n 1 / 3 ? e is P rofaabl y true for animal 

as a basis for ou ? constructi-n-^Ld /““I V 1 ?,®* 5, a PP rox ™stion to reality 
simplest assumption ^^J%^ iSed '***«**• 

associated a positive number £ such that in any operation? th'org'aTwiU 
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fail so nmctioa. correctly with the (precise) probability ^ . This malfunc¬ 
tioning is assumed to occur statistically independently of the general state 

of the network and of the occurrence of other malfunctions. A more 
general assumption, which is a good deal more realistic, is this: The mal¬ 
functions are statistically dependent on the general state of the network and 
on each other. In any particular state, however, a malfunction of the basic 

organ in question has a probability of malfunctioning which is ^ £, . For 

the present occasion, we make the first (narrower and simpler) assump¬ 
tion, and that with a single £ t Every neuron has statistically independ¬ 
ently of all else exactly the probability £ of misfiring. Evidently, it 

might as well be supposed £ f. '/a,, since an organ which consistently mis¬ 
behaves with a probability > (4, is just behaving with the negative of its 
attributed function, and a (complementary) probability of error ! /& / 
Indeed, 'if the organ is thus redefined as its own opposite, its g (> /&) 
goes then over into 5 - £. In practice it will be found necessary to 

have £ a rather small number, and one of the objectives of thinsj.investi- 
gad.on is to find the limits of this smallness, such that useful results can 
still be achieved. 

It is important to emphasize, that the difficulty introduced by ai» 
lowing error is not sn much that incorrect information will be obtained, 
but rather that irrelevant results will be produced. As a simple example, 
consider the memory organ of Figure 16. Once .stimulated, ,this network 
should .continue to emit pulses at ail later times) but suppose if has the 
probability £ of making "an error. Suppose the organ receives a stimula¬ 
tion at time t and no later ones. Let the probability that-the organ is 
still excited after -5 cycles be denoted j%, , Then the recursion formula 

. , Ps*n *• P s 4- e < i - -p s ) •>. . J 

is clearly satisfied. This can be written ....... _ 

4-j " f 5 — 3> £ } C—■ V4 } 

and so . • 

(8) p & - */* — (*,-i£5°(p 4 -i/a,)- Cf„~ ‘A) 

for small & . The quantity ■p%-*/s .can be-' taken as a rough measure of the 
amount of discrimination in the system after the S -th cycle. According 
to the above tormuia, p a -&> Vt as Co --a fact which is expressed" 

by saying that, after a long time, the memory- content of the machine dis¬ 
appears, since it tends to equal likelihood of being right or wroncr, i e g 
to irrelevancy. 

7.2 The general definition. 

This example is typical of many. In a complicated network, with 
long stimulus-response chains, the probability of errors in the basic 
organs makes the response of the final outputs unreliable, i. e, irrelevant, 
unless some control mechanism prevents the accumulation of these basic 
errors. We will consider two aspects of this problem. Let the data be 
" ejv -* , function which the automaton is to perform is given; a basic 
organ IS given (Scheffer stroke, for example); a number £ C< l /a), which 
is the probability'- of malfunctioning of this basic organ, is prescribed, 
the first question is: Given 0 >0 , can a corresponding automaton be 
constructed from the given organs, which will perform the desired func- 
tion and will commit an error (in the final result, i. e. output) with pro- 
paDility ? How small can £ be prescribed? The second question 

is. Are there other ways to interpret the problem which will allow us to 



improve the accuracy of the result,?- 




In partial anawer to fee first question! we notice now that £ , the 
prescribed maximum allowable {final} error of fee machine, must not fee 

less than g » For any output -of the automaton is fee immediate result of 
fee operation oi a single final neuron and fee reliability of fee whole system 
cannot be better than fee reliability of this last .neuron. 

7.4 The multiple line tricks 

In. answer to the second question* a method will be analysed by 
which this threshold restriction I §*■€ can be removed, m fact we will 

be able to prescribe 6 arbitrarily small (for suitable, but {feed, S }. 

The trick consists in carrying all fee messages simultaneously on fee- 
bundle az 0 lines ( M is a large integer) instead of just a single or double 

strand as in the automata described up to now. An automaton would then 
be represented by a black box wife several bundles of Inputs and outputs s 
as shown in Figure 25* Instead of requiring feat all or none of fee lines 
■of fee bundle be -stimulated, a certain critical^or fiduciary) level A is 
set: $<■£<}&* ' The simulation -of {?=■&) N lines'of a bundle is inter¬ 
preted as a positive state of the bundle. The stimulation off W -lines 
is considered as a negative state. All levels of stimulation between ' , 

these values are intermediate' or undecided. It will -fee shown'feat fey 
suitafel^c-imstructiag fee automaton/' fee number of lines deviating from • 
the '’correctly func'dssniag 5 * majorities of their bundles can be kept at or * ' 
•'below the critical level E N (wife arbitrarily high probability). Such 
a system of construction ia referred to as ''multipieadng”, Before turn- v 
lag to -the multiplexed*automata, however, it is wail to consider fee ways' 
in which error can be controlled ia our eaatomary single line networks. 

8 CONTROL- OF ERROR IN SINGLE LINE AUTOMATA 


8. 1 The simplified probability assumption. 

In 7, 3 it was indicated that when dealing with an automaton in 
which massages are carried' on a single (ox even a double) line, and 
in which the components have a definite probability & oi making an error, 
there is a lower bound to the accuracy of the operation of the machine. 

It will now be shown that it is nevertheless possible to keep the accuracy 
within reasonable bounds by suitably designing the network. For the sake 
oi simplicity only circle-Sre* automata (e£» 3.1} will he considered in 
this section, although fee conclusions -could be extended, wife proper safe¬ 
guards, to all automata. Of the various essentially equivalent, systems of 


basic organs let, section 4, 


ia the present instance, most convenient 


to select the majority organ, which is shows is- Figure 


as- fee basic 
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organ for our networks. The number £ (o < E < Vi) will denote toe probability 
each majority organ has for malfunctioning. 

8. Z The majority organ. 

We first investigate upper bounds for the probability of errors as 

impulses pass through a single majority organ of a network. Three lines 
constitute the inputs of the majority organ. They come from other organs 

or are external inputs of the network. Let ^ ^ be three numbers 

C0< 0 > which are respectively upper bounds for the probabilities that 

these lines will be carrying the wrong impulses. Then 

upper bound for the probability that the output line of the majority organ 
will act improperly. This upper bound is valid in all cases. Under proper 
circumstances it can be improved. In particular, assume: (i) The proba¬ 
bilities of errors in the input lines are independent, (ii) under proper func¬ 
tioning of the network, these lines should alwaysribkin the same state of 
excitation (either all stimulated, or all unstimulated). In this latter case 

9 ■ Mi + 1 Z.‘?4+ Ms --.2 , 2.'V‘2j> 

is an upper bound for at least two of the input lines carrying the wrong 

impulses, and thence • _ 

» Cl- S ) & t ? U- * S 4- Cl - & 

is a smaller upper bound for the probability of failure in the output line,. 

If ail. '£i' ~ *1 * then is a general upper bound,, and £+ ‘S 

S£+’l'*l a ‘ is an upper bound for the special case. Thus it appears that in the 

general case, each operation of the automaton increases the probability of• 
error, since. S+3n> tjj , .so that if the serial depth of the machine (or rather 
of.the process.to oe performed) Is very great, it will be impractical or 
impossible to,obtain any,kind o£ accuracy. In the special case, on the,other 
hand,, this is not necessarily so -• 843h is possible. Hence, the chance 
of keeping .the error under control lies in maintaining the conditions of the 
special case .throughout the construction. We will now exhibit, a method 
’which achieves this. 


8. 3 Synthesis b£ automata, 

8. 3, 1 The heuristic argument. 


The basic idea in this procedure is very simple. Instead of run¬ 
ning the incoming data into a single machine, the same information is 
simultaneously fed into a number of identical machines, and the result that 

comes out of a majority of these machines is assumed to be true. It must 

be shown that this technique can really be used to control error. 

Denote be Q ' the given network (assume two outputs in the specific 


instance pictured in Figure 26). 

copies £Tj O'* (?® respectively. 


Construct- (7 in triplicate, labeling the 

Consider the system shown in Figure 26. 



Fig, 26- 
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1 majority organs the conditions of the snsciai cass 
tain. Consequently, it X( is an upper hound fcrfche proba- 
y output of the original network Q , then 

’ J ' - h <h> 


For each, ct zh 
considered above obtain 
bilifcy of error at am 

= e + ci-aei c-Sff"- 


is an upper bound for the probability of error at any output of the new net¬ 
work 0*. The graph is the curve «ft* a £ ( :n } , shown in Figure 27. 

Consider the intersections of the curve with the diagonal 'i^ss <n 

First, *l*ZH is at any rate such an intersection. Dividing ^-f 8 C^) by ^ - \{ A 

gives CL + E ,hence the other intersections are”the roots of 

Ci“ 2 >e}^+ 6 « 0 > i. 6 , 


,«i ( l±1 j i=“ ) 


for Els '/& they do not exist (being complex (for£"> 1/4} or e ^ (for Is • 

. r y l/f 4.1. ... . - .. n. I M ' '•//■> 


while forSVVft they are | *? oC where 
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This, and the monotony and continuity of \ ■s Ofc) 


For K| sO; ^'cE 

therefore imply; ,-. 

First case,.£^ ! /i tO implies | implies VSl< $*< V 

{second case,£<'/*: o*«j< f)a •' impUea ?© $ »&«»<,-& implies'»_<»*<« 

implies. t-%• |.$.«»}<,1 .implies l“9*<»5%:n. 1 

Now we must expect numerous successive occurrences of the situa- 
tion under consideration, if it is to be used -as a basic procedure. Hence the 
iterative behavior,- of the operation^’ ;^xfe£~C*) -I is, relevant. ".Now it is clear 

trom the above, that in the first case the successive,iterates .of!the process in 
question always converge to /*-, no matter what the 'original w "{'while' in the 
second case these iterates .converge to % if the original «<yV t and to \-n 
it tee original 9 » */g 0 : ' L ' " 

In other .words; la the first case ,ao error level other than <»«. Vi can 
maintain itself in the long run. I. e, the process asymptotically degenerates 
to total irrelevance, like the one discussed in 7.1. In the second case the 
error-levels ^ ~ % and r s) <w \~% will not only maintain themselves in the 
“ on g ru J 1 * they represent the asymptotic behavior for any original 
or *)>/*, respectively. ‘ 

These arguments although heuristic, make it clear that the second 
case a.one can be used for the desired error-level control. I. e. we must 
require '/4 , i. e. the error-level for a single basic organ function must 
oe less than /o . The stable, ultimate error-level shbuldthen.be V a 
{we postulate,, of course, that the start be made with an error-level \/ A 
is smaJ if £ is, hence S must be small, and so ^ 


% 


)< 


( 11 ) 


% - £ 4 . 3£ a, -»- ... 


This would therefore give an ultimate error-level of ~ \oVk (i. e. -w ~ e f ) 
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tor a single basic organ fun 


ction err or-level of 


8. 3. 2 The rigorous argument. 

To make this heuristic argument binding, it would be necessary to 
construct an error controlling network P" for any given network y , so 

that aLl basic organs in fP n are so connected as to put them into the special 
case for a majority organ, as discussed, above. This will not be uniformly 
possible, and it will therefore be necessary to modify the above heuristic 
argument, although its general pattern will be maintained. 

Tt is, then desired, to find for any given network Jr an essentially 
equivalent network P* , which is error-safe in some suitable sense, 
that conforms with the ideas expressed so tar. We will definefchis as mean¬ 
ing , that for each output line of f n (corresponding to one of P ) ^(sep¬ 
arate) probability of an incorrect message (over this line) is * ft , The 
value of % will result from the subsequent discussion. 

The construction -will be an induction over the longest serial chain 
of basic organs in f* , say pis CP ). 

Consider the structure of P . The number of its inputs i and out- ' 
puts or is arbitrary, but every output of f* must either come from a basic, 
organ in 5® , or directly from an input, ‘‘or from a ground or live source./ .■ 
Omit the first, mentioned basic organs from P , as well as the outputs 
other than the first mentioned ones, and designate the network that is lefty 
over by Q. . This is schematically shown in Figure 28. (Some of the’ ,J 
apparently separate outputs of Q. may be split lines coming from a single 
one, but this is irrelevant for what follows,) • ' " ! 

If Q is void, then there is nothing to prove; let therefore Q, be hop- 
void. Then clearly {•*■ i eSL 1 s, Cl 3 )-1. 

Hence the induction permits us to assume the existence of a net-' -‘v 
work Q,* which is essentially equivalent to GL , and has for each’output ->■' 
a (separate) error-probability *£ 5 Hi * ■ 

We now provide three copies of Q'“ : Q, y Q. j Q, ' i &nd con¬ 
struct P* as shown in Figure 29.- (Instead of drawing the, rather com¬ 
plicated, connections across the two dotted areas, they are indicated by 
attaching identical markings to endings that should be connected, ) 

Mow the (separate) output error-probabilities of & are (by in¬ 
ductive assumption) ■& . The majority organs in the first column in 

the above figure (those without a ® ) are so connected as to belong into the 
special case for a majority organ (cf. 8. 2), hence their outputs have (sep¬ 
arate) error-probabilities “£. 3^00 * The majority organs in the second 

column in the above figure (those'with a Q ) are in the general case, hence 
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their (separate) error-probabilities are -s, £, 4 .1 £ Cty} 

Consequently the inductive step succeeds, and therefore the attempted 
inductive proof is binding-, if 

(12) £4-3 f s s TJi, , 

8,4 Numerical evaluation. 

Substituting the expression (9) for f % £•«) into condition (12) gives 

*4- l 4= 3 Cl «• O.S'iC 3 “2. 'pf > < T), ? 
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Clearly the smallest fulfilling this condition is wanted. Since the left 

hand side is •< 0 for >o this means the smallest (real, and hence* by 

the abcive, positive) root of 


(13) 


V, 
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We know from the preceding heuristic argument, that will be necessary 

-- but actually even more must be required. Indeed* for «j l= */a the left 
hand side of (13) is » — 0 + 0/hence a significant and acceptable 
rf is (i. s: ani ‘/a. )*can fee obtained from (13) only if it has three real roots, 

A simple'calculation sh6iVs, that for £ s vt. only one real root exists n, *,42 .y. 
Hence the limiting £ calls for the existence of a double root. Further cal¬ 
culations^ shows, that the double roof in question occurs for £ « .0073, and 
that its value is vj, ss . 060, Consequently 6 < , 0073 is the actual requirement 
i. e. the error-level of a single basic organ function must be «s£ . 73 %. (• . ■. •• 
The stable, ultimate error-level is then the smallest positive root ■#, of (13), 
y|j is small if E, is, hence £ must be small, and so (from (13)) 

\i ~ V 152. &H 

It is easily seen, that e. g. an ultimate error level of 2% (i. e„ P, s ,©£ ) calls 

for a single basic organ function error-level of . 41% (i s e. £ s? , 0041), 

This result shows that errors can be controlled. But the method of 
construction used in the proof ahput threefolds the number of basic organs in 
~r for an increase of £*-(&) by 1, hence..S 5 * has to contain about 3 f* 
such organs. Consequently the procedure is impractical. 

The restriction £ < „ 0073 has no absolute significance. It could be 
relaxed by iterating the process of triplication at each step. The inequality 
£ <Z» is essential, however, since our first argument showed, that for & % t/s 
even for a basic organ in the most favorable situation (namely in the "special 11 
one) no interval of improvement- exists. 


9 THE TECHNIQUE OF MULTIPLEXING. 


9c 1 General remarks on multiplexing. 

The general process of multiplexing in order to control error was 
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already referred to in 7,4. The messages are carried on N lines. A posi¬ 
tive A '-A* '/x> 3.s chosen, and £xr^*.-o_£.x,ox ot x X v lines ot 

bundle is interpreted as a positive message, the stimulation oi sJ 4iJvf lines 
as a negates message. Any other number of stimulated lines is interprets 
•ed as malfunction, The complete system must be organised is such a man¬ 
ner, that a T&ai&mctioia of the whole automates cannot he caused by the mal¬ 
functioning of a single component, or of a small number of components , but 
only by the malfunctioning of a large number of them. As we will see later, 
the probability of such occurrences can. hs made arbitrarily small provided 
the number of Hues in each bundle is made sufficiently great. All of section 
9 will be devoted to a description of the method oi constructing multiplexed 
automata and its discussion, without considering tha possibility of error in 
the basic components. In section 10 we will fees introduce errors in the 
basic components, and estimate their effects. 


9. 2 The majority organ. 


I The basis executive organ. 


The first thing to consider is the method of constructing networks 
which will perform the tasks of th® basic organs for bundles of inputs sad 

outputs instead of single Hues, . . 

_ A simple example will make the.process clear. ’C&tsider. the pro¬ 
blem of constructing the analogue of the majority organ which will accomo¬ 
date bundles of five lines. This is easily done using the .ordinary majority 
organ of Figure 14, as shown in Figure SO.' fTtie'cotmeo^oas are replaced 
fey suitable markings.* in She same’ way-as is Figure '1 ?*)’ .?' ’ ‘V ' '■*” • 

9.2,2 The need for a restoring organ. ".V * ; 


It is intuitively clear feat U almost all lines of W© ©2 the' input 

bundles are stimulated, idea afejheai all lines' oi fee output bundle will fee 

— 3 ... A.... £9a ? « ' . _ ' _ « «- * * „ »• « . '* 



oi it® output line© 


ever, anofes: 


fast is brought id ItaM, 


Suppose 


feas a crttacai level^aV^ is set Cor the bundles. Then if Wo of fee input 

bundles have 4,lines stimulated while the other has abne f the output may 
have only 3 lines stimulated. The same effect prevails in fee negative 


the 


third 

suiting output may be 


r ‘sse. If two bundles have just one input each stimulated, 
bundle lm,s all of its inputs stimulated, then fee resulting__ 

the stimulation of two lines. In other words, the relative number of 

lines in the bundle- which are not is the majority state, can double in 

passing through, fee generalised majority system, A more careful ana 
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ysis (simuar to tat one that will be gone into in more detail for the ca^ 

of the Scheffer organ in 10) shows the following: If, in some situation the 
ope*avion of tne organ snould be governed by a two-to-one materi+v n/the 
input bundles (/, e. if two of these bundles are both prevalently stimulated 
or both prevalently non-stimulaied, while the third one is in ' 1 ., ed 

«*«”«•» probable level of the j wmbf^S^x. 

" s “ m ~ -if, errors m the tv/o governing input bundles; orfthe 
? tl , , h d ' " n an operation in which the organ is governed by a unanimous 
behavior of its input bundles (i. c. if all three of these bundles are or^al 
ently stimulated or all three are prevalently non-stimulated), then the out- 

Thi'J r *n°rt ‘ g ^ raUy be Smalier than the (maximum of the) input errors 
Inns in the significant case of two-to-one majorization, two significant 

ft ° P ;° dUCe a reSUlt lyin § in the intermediateS o£ 

uncertain information. What is needed therefore, is a new type of or 

which will restore the original stimulation level. In other v/Srds we?eed 

Uteantf ing . th * f r °P e f ty that ' with a fairly high degree of probability 
*t transforms an input bundle with a stimulation level which is nLr to zero 

or to one mto an output bundle with stimulation level which is even c iose- 
to the corresponding extreme. even closer 

T{lo Ph f' s tbe ^ u itiplexed systems must contain two types of organs 
The Ur at type is the executive organ which performs the desired basic” 
operations on tee oundi.es. The second type is an organ which -restores 

fausS ? 11 l * Vel .. 0f the bundl8S * and b *«ce erase! the degradation 
caused by she executive organs,- This’ situation has its ahSaiosr in m&nv of 

° mata f hiCh Pe?fc?m io ® isaliy implicated S fS Z “• 
ample in electrical circuits, some of the vacuum tubes perform executive 

Srx 1 .;’ detection or rectification or gotoing c”Zci£nZ 

i" a “ 1Sned ‘ he # “ 5k which 

9.2.3 The restoring organ, ■ ■ . , f 

9. 2, 3, l Construction. 

and * n -^ e ^ S fr- ti0n ° f a reSt ° ring organ is 9 uite simple in principle, 

J *n fact contained xn she second remark made in 9, 2, 2, In a~cvud- wav 

the ordinary majority organ already performs this task. Indeed in the 
c S'LVrilSf * bundle oi to. line*, the majority or^n has pre - 

“'ut .‘liaolt Wo f ' 5 11 s, *PP re »s«* a single incoming impulse 

as weu as a single incoming non-impulse, i, e . it amplifies the prevalence 

trait mJft S 'W-fv ^ «- ^ ab * ence of im Pnlses. To display this . “ ' 
shoi^Tn Fi^uri 3i. " t0 SpUt its ou ^ ut line ^to three lines, as 

above rnZ f° r . large fcuT f es « ^ ***** of the remark referred to 
ed K v ' ™l,T ng l h * rsduct7 f n o[ errors in the case of a response induc¬ 
ed hy a unanimous behavior of the input bundles, it is possible to connect 

P&r&llel aGd th£?£by the P de^ed remora- 

* Howevcr ‘ 14 1S necessary to assume that the stimulated (or non-stim- 
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utated) lines are distributed, at random in the bundle. This randomness must 
then be maintained at ait times. The principle is illustrated by Figure 32. 
The ’’black box"U is supposed to permute the lines of the input bundle that 
pass through it? so as to restore the randomness of the pulses in. its lines. 
This is necessary, since to the left o£ H the input bundle consists of a set of 
triads, where the lines of each triad originate in the splitting at a single line, 
and hence are always all three in the same condition. Yet- to the right of 14* 
the lines of the corresponding triad must be statistically independents in 
order to permit the application -of the statistical formula, to he given below for 
the functioning of the majority organ into which they feed. The way to select 
such a ’’randomizing 51 permutation will sot be considered here -- it is intuit¬ 
ively plausible that most ” complicated” permutations will be suited for this 
’’randomizing” role. (Of, 11.2.) 


9. 2. 3. 2 Numerical evaluation. 

If #ij¥ of the A?* incoming lines are stimulated, then the probability of 
any majority organ being stimulated (by two or three stimulated inputs) is 


(14) 


ft Of' 


This approximately (i, e, with high probability, provided// is large) ** & M 
outputs will be excited. Plotting the curve of ©T* against d , S3 shown im 
Figure t ha t this organ wilt have tS&e desired character¬ 
istics:’ '.Zi ' V p ";. ’ 

This ojhye inters sets the 'diagonal M -s'g three 'times/ ForOCs ^-^1 . 
0461 impliesI 1&<st< I implies I, X e» successive iterates 

of this process converge to f? if the original and to I if the original 

$»vk. - " *; ’ ’ 

In other .words: .The error levels&nd0f«*I will not -only main- • 
tain themselves rathe long run. but they represent the', a ay mptotic behavior 

for any originalef«or eSi>¥& » respectively. Not®, that because of 
9 1 • there is'complete symmetry between'the 5 /&region and the'" 

region* 

The process-thus brings every SI nearer to that one of $ and 
I , to which it was nearer originally. This is precisely that process of 

restoration, which was seen in 9. 2, Z to be necessary. I, e. one or more 
(successive) applications of this process will have the required restoring 

effect, ' ^ 


ThenSC^ ©f^a Ls, the maximum 

restoration, is effected on error levels at the distance of 21. 2% from |0* 
or 100% -- these are improved (brought nearer) by 9.6%. 

f 1 
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9- 3 Other basic organs. 

We have so far assumed that the basic components of the construct¬ 
ion are majority organs. From these, an analogue of the majority organ -- 
one which picked out a majority of bundles instead of a majority of single 

lines -- was constructed. Since this, when viewed as a basic organ, is a 

universal organ, these considerations show that it is at least theoretically 
possible to construct any network with bundles instead of single Hues, How¬ 
ever there was no necessity for starting from majority organs. Indeed, any 
other basic system whose universality was established in section 4 can be 
used instead. The simplest procedure in such a case is to construct an 
(essential) equivalent of the (single line) majority organ from the given basic 
system (cf. 4. 2. 2), and then, proceed with this composite majority organ in 
the same way, as was doue above with the basic majority organ. 

Thus, it' the basic organs are those Nos, one and two in Figure 10 
(cf. the relevant discussion in 4, 1, Z), then the basic synthesis (that of the 
majority organ, cf. above) is immediately derivable from the introductory 
formula of Figure 14. 

9.4 The Scheffer stroke. 

9,4. .1 The. executive organ. 

Similarly, it is possible to construct the entire mechanism start¬ 
ing from the Scheffer organ of Figure 12, In this ease, however, it is 
simpler not to effect the passage to an (essential) equivalent of the major¬ 
ity organ (as suggested above), but to start de novo. Actually, the same 
procedure, which was seen above to work for the majority "organ, ’ works 
mutatis mutandis for the Scheffer organ,too, A brief description of the 
direct procedure in this case is given in what follows; 

Again, one begins by constructing a network which will perform the 
task of the Scheffer organ for bundles of inputs and outputs instead of single 
lines. This is shown in Figure 34 for bundles of live wires, (The connect¬ 
ions are replaced by suitable markings, as in Figures29 and 30.) 

It is intuitively clear that if almost all lines of both input bundles 
are stimulated, then almost none of the lines of the output bundle will be 
stimulated. Similarly, if almost none of the lines of one input bundle are 
stimulated, then almost all lines of the output bundle will be stimulated. 

In addition to this overall behavior, the following detailed behavior is 
found (cf. the detailed consideration in 10. 4). If the condition of the organ 
is one of prevalent non-stimulation of the output bundle, and hence is 
governed by (prevalent stimulation of) both, input bundles, then the most 
probable level of the output error will be (approximately) the sum of the 
errors in the two governing input bundles; if on the other hand the condi¬ 
tion of the organ is one of prevalent stimulation of the output bundle, and 
hence is governed by (prevalent non-stimulation of) one or of both input 
bundles, then the output error will be on (approximately) the same level 


Fig. 34 








as the input error- if (only) one input handle Is governing (i, e. prevaieaS&y 
non-stimulated!. and it will be generally smaller than the Input error? if 
both input bundles are governing (L e s prevalently non-sfcimulaied}* Thus 

two significant iaputo may produce a result lying Is fee istermsdiete Bone 

error lovsll is 


aac«?tsa mtermaaos* nence a 


rnslesrfet orgats 


l_ S .-y,. 
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2 The restoring organ. 


Again,- the above indicates that the restoring organ can be obtained 
from a special case functioning of the standard executive organ? namely by 
obtaining’ all itspufcs from a single input bundle? and seeing to it feat fee out¬ 
put bundle has the same s’iae as the original input bundle. The principle is 
illustrated by Figure 35. The M f>Ls.ck bC8r M M is again supposed to effect a 
suitable permutation of the lines that pass through it? for the same reasons 
and in the same manner as in the corresponding situation for the majority 
organ (cs. Figure 32), X,e« it must have a a sa,nSmm%fs^ , effect. 

If Of#’of the .Af incoming lings are stimulated, Sues the parofeeMtity 
of any Scheffer organ feeing stimulated (by at least one non-stimulated in¬ 
put) is 


<1 


-«■?» § 


r*K* 


via \ 

4 , 


Thus approximately (i, e, with high probability provided M is Large) M ■ 
outputs will bh excited. Plotting the curve, of ASr against SC discloses some ' 
characteristic differences against fee previous case (feci case ©2 -fee majority 
organs, i« e.€f Si as cf, 9,2,3), which requirs farfeer discos "• . 

sion. This curve is shown in Figure 36, Clearly ©f* is an antimonotone 
function of ©f 9 i, e, instead ©£ restoring an excitation level {£, e, bringing 
it closer to 0 or to ! , respectively), it transforms it into its opposite 
(i, e, it brings fee neighborhood of £' close to $ , sad the neighborhood of 
| close to © ), In addition it produces tor Of near to I an less£ 
near to 0 (about twice farther), but for sC near to & an c^’much nearer 
to $ (second order!). All these circumstances suggest, that fee operation 
should fee iterated, 

.Let the restoring organ therefore consist of two of the previously 
pictured organs in series, as -shown in Figure 37, (The ‘'black boxes" 

2i f g 1 % play fee same role as their analog 1SI plays in Figure 35,) TMs 


organ trcssiorms aa input 'sxcis&sisa 


-vaT®sj? nte 


level of approximately (cf, above}'v®, wh« 
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his curve is very 
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This curve of ©T T against« is shown in Figure 3„. 
similar to that one obtained for the majority organ (i 
cf„ ), Indeed: The curve intersects the diagonal ^ in 

the interval 0 * «<4 1 three times: For 0 , si^ J I, where os, £- I «i- ig j/£, a 
. s & * (There is a fourth intersection et sr-l-o^ « -USSff, but this is 

irrelevant, since it'is not in the interval 0< e«K implies 

0 <« <o( ; I implies 0< <#*'*< i. * 

In other,words; The role of the'error levels Kvfia nd Ot<v- i is pre¬ 
cisely the same as for the majority organ (cf, 9 . 2 , 3 ), except that the limit 
between then* respective areas of control lies at oi a £4 instead of at «**/& 
I.e. the process 0(-* brings every ai nearer to eitherfO or to I , but 
she preference to Q or to I is settled at a discrimination'level of 6 i e $% 

(i» e. ) instead os one of 50% (i. e.i/2) s Thus, apart from a certain 
asymmetric distortion, ihe-4fs?g&n behaves like its counterpart considered 
tor the majority organ --,i. e, it is an effective restoring mechanism." 

10 ERROR IN MULTIPLEX SYSTEMS, 

10 . i General remarks. 

. ln section 9 the technique for constructing multiplexed automata 
was described./;;However, .the role of errors entered at best intuitively 
&tKx summarily*-.. and ih&raforo *it has a till not proved that thftsg 

systems will do what is claimed for them -- namely control error 

taon 10 is devoted to a sketch ?f the statistical analysis necessary to show 
tuat, bv using large enough bundles of, lines, any desired degree of ac¬ 
curacy \i, e. as small a probability "of malfunction of the ultimate output 
of the network as desired) can be obtained vti.ih a multiplexed automaton, 
or simplicity, we will only consider automata which are con » 
strutted tram the Scheffer organs. These are easier to analyse since 
they involve only two inputs. At the same time, the Scheffer organ is 
(by itstell) universal (cf. 4.2, 1), hence every automaton is essentially 
equivalent to a network of Scheffer organs, 

Errors in the operation of an automaton arise from two sources, 

F lrst > the individual basic organs can make mistakes. It will be as¬ 
sumed as before, that, under any circumstance, the probability of this 

happening is just g , Any operation on the bundle can be considered as 
a random sampling of size/v (jV being the size of the bundle). The 

^ mber « 0f t ri 'V S COmmitted fa ^ ths individual basic organs in any opera- 

tiou on the bundle is then a random variable, dis tributed ap proximately 
normally with mean eff and standard deviation l/eCt- t) A second 

faiIure f f ris 5 * because in operating with bundles which are not 


same state of stimulation or non- 
<nA* 


Fig, 


stimulation, the possibility of 
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multiplying error by unfortunate combinations of lines into the basic (single 
line) organs is always present. This interacts with the statistical effects, 
and in particular with the processes of degeneration and of restoration of 
which we spoke in 9. Z, Z, 9. Z, 2 and 9. 4. 2. 

10. 2 The distribution of the response set size. 

10. 2. 1 Exact theory. 


In order to give a statistical treatment of the problem, consider the 

Figure 34, showing a network of Scheffer organs, which was discussed in 

9.4. 1. Let again.//be the number of lines in each (input or output) bundle. 
Let X be the set of those &.s for which line No. i in the first input 

bundle is stimulated at time -fc ; let Y be the corresponding set for the 

second input bundle and time t ; and let 2 be the corresponding set for the 

output bundle, assuming the correct functioning of all the Scheffer organs 
ut*ea, e sl.emencs ? respectively* out 

otherwise be random -- i. e, equidistributed'dver all pairs of sets with these 
numbers of elements. ^What can then be said about the number of elements 
1^" of £ ? Clearly f £ are the relative levels of excitation.of the two 
input bundles and of the output bundle,' respectively, of the‘network under 
consideration. The question is then; What is the distribution of the (stoch¬ 
astic) variable ^ in terms of the (given) S 4 Yi ? - 

LetlAT be the. complementary set of H . Let p, Q, r be the numbers 
of elements oiX_, X, W) , respectively, so that ps| 4 
Then the problem is to'determine the distribution of the (Stodhasttc) Vari- 
sbls f* in tfirrns £y£ iijh.e (*/?» * «.«=. i. ■§■-?% & * v**® ? •? ^ 


w 15 aeany me intersection ©t me sets: a. i lWs^i # Let O, V 
be the (relative) complements of>/in X,T respectively;' U»X-W,y=y*.V 

and let 3 be the (absolute, i. e. in the set (l. am . complement of the * 

sum of JC andY • .&* - CX-bY) . Then"W£ lij'VjS are pairwise* disjoint sets 

making up together precisely the entire set ( i s .. a 9 iv r j 'wffi r, p-r. a - K 
elements, respectively. Apart from this they areYrarestricted. 
Thus they offer together Nl /M Cp-nl c^-r)» (N-P-V-rJ!! possible choices. 
Since there are a priori Wl / C pi <N'-p>i3 possible choices of an X with p 

elements and a priori Kl»/ £ql Crt-^pl 3 possible choices of a. Y with o 
elements, this means that the lookeu for probability of haviuP de — 
meats is ' ‘ '• ■ ~® 
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'• Cp-r/j 7N r -p-^Yrii wT 


Lote, that this formula also shows that 0 when i“<0 or p-Y*<C0 

or d- r<0 or l\|-r<0ji» e. when T violates the conditions 

Max (0, p+$- W) < r« Kin. 

This is clear combinatorially, in view of the meaning of jC,*\ and y/» In 
terms of !„*!£>’S the above conditions become 
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its entire interval'of variaMUtv acce^4iaf_ So (IT) ). Geaaeqw 
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for ail i»f^{w4tMn the interval ' {!?) )<, This implies ~ s ih view 

(It) th &I for all f wMeh are signilie&f&fciy^ C* f *|, ^ fends I© U very ' 

ra^MLy as M gets large. It aofficea therefore to*evuiuate (It) for f •**' 
i-|^. Now a« t/f (t»|) a £i,p^f<rJ"» l/tf 0*1$ ^ (j« »)1 
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is an acceptable approximation for p . _ „ 

V* is an late ger - valued variable, hence € s 1 — _fr is a rational¬ 
valued variable, with the fixed denominator J'v . Since Jtf’is assumed to be 
very large, the range at f is very dense* If is therefore permissible to 
replace it by a continuous one, and to describe the distribution of f by a 
probability-density 0 s » p is the probability of a single value of ^ , and 
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since the values of are equidistant, with, a separation cL F ~ i/N th* * 

lation between CT and 9 is best defined by crdL^e , £ e ^ 

fore (19) becomes s " 5 ' CT a pJv , There- 

(■/()] rr 1 -if _ _ LLzAlll ) a 

‘vGF Tfnfff^TTF^' e 

This formula means obviously the following* 

а , J. is with the mean i-£, 

and the dispersion \/Tr, I gT^TT ITwUp -m * at. it , ? 1 

fwhi'ch^^^^onentia^^is 1 valid^^^l^on^ as^^ 

efficient of JT (in the exponent, L e . -I ( rt ,* ^ co “ 

is somewhat altered as 5, deviates from" \- & J “ kh^lU^V ^’ ^ } 
discussion of & given above.) ~ 11 * ( Thls follows from the 

Tli.f? simple sts.tistic s.^ rH^^si r\-*^ <-> £ o a . , . 

б, the unique Value U . We see "new thkttH «.*“•' dmounte f to attributing to 

1 ~ ce no,w lnat this is approximately truer 
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(Z!) j^is a stochastic variable, normally distributed, with the mean 
1.0 and the dispersion i e meaTi 

10 . £.2 Theory with errors. 

si f l 3 SF e ~ 

is?i - 

ber of errors her® is anDrov-mafei,, S1 ,.• L;/ one UTil *. The num- 

er'and the dispersion •fh^F ' LTSt 7 dlat J lout f d [. v/ith the mean 

the N-r organs, each ofVhich should correctW s^rul'b ^ . Amon S 
each error increases r* by one unit The 1 / Ute lfcs out P u t, 

.pp«*U»UI T normally distributed: wik '5” r i, here *•'»«•*» 

and the dispersion fT TT ^T i?u-r? (. r * uv V 

terence of these two (independent) siochasHc^^ e ) Thus r'-r is the dif- 

approximately normally distributed with th vari ables, - Hence at, too, is 
and the dispersion distributed, with the mean - S r 4 € (N-rj« t(tt-zr), 
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normally distributed, with the mean 0 and the dispersion I 
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and then by (21) 
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too, is normally distributed, with the mean 0 and the dispersion 
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Hence (21) becomes at last (we write again f in place of f * ); 
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6 is a stochastic variable, normally distributed, with 

the mean 0 and the dispersion \ . 


10, 3 The restoring organ, 

J -^ s discussion equally covers the situations that are dealt with in 
i igures 35 and o7, showing networks of Scheffer organs in 9. 4, 2, 

Consider first Figure 35. We have here a single input bundle of Jf 
lines, and an output bundle o ij\f lines. However, the two-way split and 
the subsequent '' randomizing" permutation produce an input bundle of Qjf 
lines and (to the right of X£ ) the even lines of this bundle on one hand, 
and its odd lines on'the other hand, may be viewed as two input bundles 
of lines each. Beyond this point the network is the same as that one - 
of Figure 34, discussed in 9. 4. 1. If the original input bundle had 
stimulated lines, then each one of the two derived input bundles will also 
nave g Jf stimulated lines. (To be sure of this, it is necessary to choose 
t.iS randomizing' 5 permutation Ui of Figure 35 in such a manner, that it 
permutes the even lines among each other, and the odd lines among each 
other. This is compatible with its "randomizing" the relationship of the 
family of all even lines to the family of all odd lines. Hence it is rea¬ 
sonable to expect, that this requirement does not conflict with the desired 

F» T 5' K *?. m * Z * n ® n c ^ arac *’ er permutation,) Let the output bundle have 

F, stimulated lines. Then we are clearly dealing with the same case as 
in (22), except that it is specialized to f — T) . 
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Hence (22) becomes: 

f 


(23) 
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o is a stochastic variable. 

, 0 and the dispersion $ 


normally distributed, with the mean 


Consider next Figure 37. Three bundles are relevant here- The 
mput bundle at the extreme left, the intermediate bundle issuing directly 
from the first tier of Scheffer organs, and the output bundle, issuing dir 
ecily from the second tier of Scheffer organs, i. e! at the extreme right 

h , , e ,° f .. thes ® three bundles consists ofJ\Tlines» Let the numbe/o/ 
stimulated lines m each bundle be P/T \pjF reaoectiveiv tu,. /ju 
above applies, with,,, ».< C« “ 

W * C\-^) + as Cf-i) f J 

W (O-asJ* ao-f))S £ 0-£))/AT £** 

. ^ 


(M)<- 


^ -S C i - cb""} -h 2 g < CO 2 * - -b ) + 


+ V Ccwia) 8 - £*** 

c** r*#* 

^ , , are stochastic variable? independently and normally 
distributed, with the mean 0 and the dispersion f . Y 

10.4 Qualitative evaluation of the results. 


ilevant 


i- e/ the Schefft 


In what follows, (22) and (24) wiH 
organ networks of Figures 34 and 37. 

Before going into these considerations however we have t n m ,i, 

an observation c once mine f?2) (?7\ ch-va,-* s > “? ve make 

levels § 5 <o on the input bundle* Xr*l h "T° f*’ the ( reiativ */ excitation 

<i. e« for large JN~ and^sm-• ■' e 1 / network generate approximately 

on the outeut h„Li & J the Relative) excitation ievel\e a « | - fl 

m% l l Ltaitri ™ 5 «“ **at«5at. mile 

two input bundle, are both pr.vnlenU® stimjuted f 1 if f t „ 

then the distance of ?. from 0 U ah^ut hLA* e **\. P .V < » * : 

and of from I v ?"■ /, . * um ot the ^stances of f 

11 . abS ms.ij~£? r.k the I ?i si s i ” •*f. <» 

prevalently non-stimulated. i e if £ 'tT Is both input bundles 
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X0-. 5 Complete quantitative theory. 
ID, 5, 1 General results. 


W& c&~a now pass to the complete statistical analysis of the Scheffer 
stroke operation on bundles. In order to do this, we must agree on a sy¬ 
stematic "way to handle this operasiou by & netwoi'iC ihe system to ho adop't“ 
ed will be the following: The necessary executive organ will be followed in 
series by a restoring organ,. I. e, the Scheffer organ network of Figure 34 
will be followed in series by the Sc heifer organ network of Figure 37. This 
means that the formulas of* (22) are to he followed by those cr£ (24), Thus 
%.,T! are the excitation levels of the two input bundles, V' is the excitation 
level of the output bundle, and we have; 


C»- % <» 
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1 0,0 ? fi"‘ are stochastic variable®, independently and aor- 

jmaliy distributed, with the mean 0 and the dispersion 
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Consider now & giver* Educmry level A 


Then we need a be¬ 


havior, like the "correct* 1 one of the Scheffer stroke, with an overwhelm¬ 


ing probability. This msaasi ,Th® isnplicaUoa of 
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may, ot course, only se expectea tcryy sutncieimy targe and 
«£. sufficiently small. In addition, it will be necessary to make an ap¬ 
propriate choice of the fiduciary level £& . 

3fj\ r is so large and -g is so small, that all terms in (25) contain¬ 
ing factors a-«d & can be neglected, then the above desired rl over¬ 

whelmingly probable** inferences become even strictly true, if A i s small , 


enough. Indeed, then { 
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— Now it is easy to verify tyssQ for § > 

'*'*1 VX C . /. - - /sSUl - ,* _ A ? 


Hence sufficiently smal 


V'* /-O for 

y small fL will guarantee the desiderata stated fur 
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urtaer aoove 
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10, 5. 2 Numerical evaluation. 


Consider next the case of a fixed, finite Jf and a fixed, positive <£ . 

Then a more elaborate calculation must be based on the complete formulae 
of (25). This calculation will not be carried -out here, but its results will 
be described. 

The most favorable fiduciary level A , from the point of view of 
this calculation turns out to be A = ,07 , L e, stimulation of at least 

93% of the lines of a bundle represents a positive message; -stimulation 
of at most 7% of the lines of a bundle represents a negative message; the 
interval between 7% and 93% is a zone of uncertainty, indicating an effect¬ 
ive malfunction of the network. 

Having established this fiduciary level, there exists also an upper 
bound for the allowable values of 8 . This is £ ® ,0107. In other words-, 
if £ *2 . 0107; the-risk of effective malfunction of the network will be above 

a fixed, positive lower bound, no matter how large a bundle size Mis used. 

The calculations were therefore continued with a specific £<,0107, namely, 
with 8 s . 005. 

With these assumptions, then, the calculation yields an estimate for 

the probability of malfunction of the network, i. e. of the violation of the 

desiderata stated further above. As is to be expected,' this estimate is 
given by an error integral. This is 


(26) 


pen) a [ £ 

J fair J K 




ctx 


< t-s 
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expresses,'-m a certain sense, the total allowable error divided by a com¬ 
posite standard deviation. The approximation is of course valid only for 

larg ejf,. It can also be written in the form 


. g6>jy 

(27) per,) ss ~ (O* u> - 

v A/ ® 

The following table gives a better idea of the dependency expressed by the 
formula; •' 



n = number of lines in a bundle 


jp(n)= probability of malfunction 


1,000 

2, 

*7 

' % n“2 

-■2. ia 

2, 000 

l 

2. 

6 

X 

- 3 

10 J 

3, 000 | 

2. 

s 

a 

XlL 

10‘ 4 

5, 000 


4 

X 

10“ 6 

10,000 ; 

1. 

! 

6 

X 

o 

6 

O 

20,000 

Zo 

8 

X 

10"19 

25, 000 

« 

1, 

& 

X 

10*' 23 


Notice that for as many as 1000 lines in a bundle, the reliability (about 

3%) is rather poor. (Indeed, it is inferior to the £ = .005, i. e. 1/2%, 

that we started with.) However, a 25 fold increase in this size gives 

”Siy u x cudyiii.tye ' 

10. 5. 3 Examples. 

10,5,3.1 First, example. 

To get an idea s£ the .significance ot these sizes and the cor- 
responding approximations, consider the two following examples, : ' 1 

Consider first a computing machine with 2500 vacuum’tubes; 
each of which is actuated on the average once every 5 microseconds. 
Assume that & mean free path of 3 hours between errors 1® desired. la 
this period of time there will have bees' £ x 'tfado a 8 « s^eo * to* ' 

* 1,4* te*"" actuations, hence the above specification calls for 
£ y i/Cu-4* SO 3 3 e7* 10“ ' . . • According to the above table 

his calls iox as Jv between iQ,GQQ and 20, 000 = ■» interpblaiing Linearly 
on — fog d gives AT = 14, 000. I. e. the system should be multi¬ 
plexed 14, 000 times. 

Jt is characteristic for the steepness of statistical curves in 
this domain of large numbers of events, that a 25 percent increase of 

J* , i.e. JV = 17,500, gives (again by interpolation) S « 4.5x10" 17, 

i, e. a reliability which is 1, 600 times better. 

10. 5.3.2 Second example. 


Consider second a plausible quantitative picture for the func¬ 
tioning ox the human nervous system. The number of neurons involved 
is usually given as i0 iV , but this number may be somewhat low, also fch 
synaptic end-bulbs and other possible autonomous sub-units may in¬ 
crease it significantly, perhaps a few hundred times. Eet us therefore 
use the figure 10 - for the number of basic organs that are present, A 
neuron may be actuated up to 200 times per second, but this is an ab» 
normally high rate of actuation. The average neuron will probably be 
actuated a good deal less frequently, in the absence oi better informa¬ 
tion 10 actuations per second may be taken as an average figurd of at 
least the rioht ord*r. Tf -U V>a>><4 f.-> toll *u~. *.___ 

^ . -• ** »***-&. w tw i3v--c.i £i.tc cl li. la p. 
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errors should be. Let us take the view that errors properly defined are 
to be quite serious errors, and since they are not ordinarily observed, 

let us take a mean free path which is long compared to an ordinary human 

life -- say 10.000 years. This means 1G i - x 10,000x 31, 536, 000 x 10 a 
- L 2 x 10^5 actuations, hence it calls tor £ — i/Cs.&K iQ iS ) - 2 x 

10“ . According to the table this lies somewhat beyond $*25, 000 

extrapolating linearly on -- 10 logS glares h f & 28, 000, 

Note, that if this interpretation of the functioning of the human 
nervous system were a valid one (for this cf, the remark of 11, 1), the 
number of basic organs involved would have to be reduced by a factor 

28, 000. This reduces the number of relevant actuations and increases the 

value of the necessary £ by the same factor. I. e, £ <= 9 x 10-22., anc j. 

hence W = 23,000. ' The reduction of Jf is remarkably small -- only 20%! 
This makes a reevaluation of the reduced Jf with the new^juoneces^ 

sary: In tact the new factor, i. e. 23,000, gives £* 7. 4 x 10 -22 *hd this 

with the approximation used above, again M~ 23, 000. (Actually the change 
of Jf is —*120, i. e. only i/2%!) 

Replacing the 10, 000 years, used above rather arbitrarily, by 

6 months, introduces another factor 20, 000, and therefore a change of 
about the same size as the above one -- now the value is easily seen to be 

N s 23,000 (uncorrected} or Af s*. 19,-000 (corrected)* 

10, 6 Conclusions, 


All this shows, that the order of magnitude of tf is remarkably 
insensitive to variations in the requirements, as long as these reouire- 
ments are rather exacting ones, but not wholly outside the range of our 

(industrial or .natural) experience. Indeed, the Jf obtained above were all 

20, 000, to within variations lying between -30% and 4 40%, 


10,7 The general scheme of multiplexing. 

This is an opportune place to summarise our results concerning 
multiplexing, i. e. the sections 9 and 10, Suppose it is desired, to build 


a machine to perform the logical function fix. 


~ -- j < ^ tj|'£ * » « f w*WJ Si gIVCii 

accuracy (probability of malfunction on the final result of the entire opera¬ 
tion) p , using Scheffer neurons whose reliability (or accuracy), i. e. 


with a given 


probability of malfunction on 


?! p 


Lr'pQ r&tionj 


5 «! V s 


We assume t 


-• The procedure is then as follows. 

First, design a network R for this function -f C ) 
as though the basic (Scheffer) organs had perfect accuracy. Second, 
estimate the maximum number of single (perfect^ 1 Scheffer organ react¬ 
ions (summed over all successive operations of ail the Scheffer organs 
actually involved) that occur in the network ft in evaluating f C*, u.,... ) 
-- say m such reactions. Put & = q/m . Third, estimate the 
bundle size Jv that is needed to give the multiplexed Scheffer organ like 
network (cf. 10. 5. 2} an error probability of a*fc most 6 . Fourth, re¬ 
place each single line of the network by a bundle of size Jf , and each 
Scheffer neuron of the network & by the multiplexed Scheffer organ net- 
work that goes with this M (cf. 10.5.} ) — this gives a network &<■"> . 

A "yes” will then be transmitted by the stimulation of more than 93% of 
the strands in a bundle, a "no" by the stimulation of less than 7%, and 
intermediate values will signify the occurrence of an essential malfunc¬ 
tion of the total svstem. 
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It should be noticed that this construction multiplies the number of 

lines by TV and the number of basic organs by 3/V , {in 10, 5, 3 we used a 
uniform factor of multiplication Jv . In view of the insensitivity of Jf to 
moderate changes in 6 , that we observed in 10. 5.-3. E, this difference is 

irrelevant. ) Our above considerations show, that the size of N is -w 

20, 000 xn all cases that interest us immediately. This implies, that such 
techniques are impractical for present technologies of componentry (al¬ 
though this may perhaps not be true for certain conceivable technologies of 

the future), but they are not necessarily unreasonable (at least not on 
grounds of size alone) for the micro-componentry of the human nervous 
system. 

Note, that .the conditions are significantly less favorable for the 
non-multiplexing procedure to control error described in section 8. That 
process multiplied the number of basic organs by 3^ , ft being the number 
ot consecutive steps (i.-e. basic organ actuations) from input .to output. 

(ci s the end of 8.4). (In this way of counting, iterative processes must 
be counted as many times as iterations occurs) -.Thus for Mat 160 which 
^1$?* eX f "l°S ica l depth, M even for a.conventional calculation, . 

^ ~2xl0 » i. e. somewhat above file outative .-.order of 


2x10 

electrons in t 


i* somewhat above the putative: order, of the '-number 
universe, . For. SO (only 25 percent morel) then 

’V • “ - 


1-^2 'x 10 *- - time’s more r 

ill OJjfT > 7 ? 

«s/:, 


• in' Vi sw'oi the : : ab ova• - * - 1 

:t 


, 7.0 jO 


3 &uu -v 2, 5 x -10 

this requires r hb ! comment; 

* f,'GENERAL -COMMENTS ON- DIGITALIZATION' AND* MULTIPLEXING,* 

^ii:i * 7 .'I.- ;• .? 7 - . 5 - £> *2 & H. ij. 7 } ti j ;j r, 'J ?*iii ii: ■£. > ■> ^ 

11 * a *®umpticn*regarding ; thd digital- ^ahklo’g ’? 

^“'Chardotfer'of'th-e'h&rvous^syMterh;-‘5?a >J?rdf»or,ry9}rri -»mot ® 

V*• 1(5 •Kbw'paasvts-sbme rarriark'f b£ : a Jnbregeneral-characters.-«i: 

, rhe question of- thenumber ©£T>aaic'heuronsJrequired to “build* 
a multiplexed automaton serves as an introduction for She first remark, 
fhe above discussion showsr'that the'mhltiplexing-.tethnique dr imprac- - f 
acai on the level of present technology, but quite practical for a perfect¬ 
ly conceivable, more' advanced technology, -and-for thenatural relay - 
organfc (neurbns).-'l.;e.ldt' ! hie'rely-caUs i fora 4ot-at^all uhnaturaL' ' “ 
micro- componentry. It is therefore quit^ r®iod«aV.in- ♦*>.!. —_ 

T’2i: r h U - "»<>*»•>**■*»*• « u»\ like “ 

existing human (or ratner: •'animal) nervous' system.' :: ■ . ; 

' •'.••-Tke-anSWfer xs r; ndt'Clear -but.’ ^<The : ihain' trouble with the multi- 

fowlol T 1 • a8 p . de3 . cr ^ ed ;i tl -preceding' section, As that they fol¬ 
low too^ slavishly a fixed plan of •-construction'-- and specifically one,-- - 

hiat is';mspired by-the 'conven.tionaNprocedures'of mathematics and math- ' 
aticai logics., It is true, that the animal'nervous ’systems, too, obey - 

some rigid ^architecturat'patterns in-their-large-scale construction,- andihat 
“ wse varieties, wmcirmake one suspe'cfa merely statistical design, ' 
eem to occur only in'finer detail and on-the ^micro-level. (It is char- . 

of oveJan T £ this f d t Ualifcy ’ most -investigators believe'in 'the existence 

? h Jt if 11 la . of large-ecaie nerve-stimulation and composite action 
iax. have omy a statistical character, and yet occasionally a single 
n e uron is known to control a whole reflex-arc,) Tt is true, 'that our 

multiplexing scneme, too, is rigid only in its Large-scale pattern (the 

Sve* Dhfs 6 r^X° rk ^ ’ aS ® pa ^ erc> ' and the S^eral layout of the execu- 

\ *2 - plLis 2. Sa hO titic? nrosri -n a a« a.j* A i^ i a ®? ; a ■«a =. \ _ ., 

Jf - , tt --O'™* u--vv.Pi.wu Ali, *Ut.f etua ill iu # 5, ij, Willie 

the random permutation "black boxes"'(cf. the relevant Figures 32, 
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V it 9 ‘^ 3 and 7 * 4 * 2 ) are typical Of a "merely statistical d es i Bn 

Also* S ,f ems to ba somewhat more flexibly designed, 

"analog" (ueurax) operations are rather freely alternating with 

analog (humoral) processes in their compete chain* of 

Mnahy the whole logical pattern of the ne^ous S yJtem saem^to deviate 

•i***™*Y our ordx:. 

pulse-trains that carry ° f °P e2 * a «on: . T‘ 

M^rctSSi)»VS“„ 

preselonaof’onl.Ty»ylSr3iZ?"' bU * «*- 

although miif'h mn^t. «— _^ - y, or something similar 


The 


... . -* ~r vucu 

although much more than ordinal 

m /*3 /VVbt 4. _ * _ 4.1 • /. e q * 


■y care should be exercised in passing 

=> nairas <a 1* 4-4-1-. X_ ... a. * r T. ^ 


A,„. 

(e. g. a computing macM-e' L*hL t— in an y^uraan automaton 

~ *•*““ W -‘£$3 

p«to«s ni - n f ”“»*'>*«”«“•*« ■ t.ttoMry^Jef?’ Kst«ya r«ai" S ,hL 
can lZtl7ctll IT 

11.2 Remarks concerning the concept of a random permutation. 

cerns the f.f 'f? c«- 

iniTiiTi v::™: TJtTTTVT- ln ^ 

same organ at some later h-™- *'• " , C2r ~ n or S°- n gets Dack to the 

relation. Moreover.. without randJmneL,"XatJo^^ wteVe 0 "" 

id to be amplified instead - y .. . wnere 


g. it is possibi 


errors f « u r*T .T. ia< ‘ uvinness i sltuatior 

that the V- be am P llfie d instead of cancelled out 

h.xat the machine remembers its mistakes «- *- _~ , .. - , 

perpetuates them. A simplified *??**'. *T &ereafter 

the elementary memory organ of if chls u si{ect 13 furnished by 

on the Scheffer stroke'/ ,hX f» &££ (?’ "°>A si ™ l ‘ r »“■ based 

This system, provided it make* + V’ 'If will-discuss tne latter-. 

of time. Thus it has two possibl-° jAV^ 3 , T alternate 

possible states; Either it fires at even times 




or at odd times* (For a quantitative discussion of Figure 16, cf, 7, 1,} 
However, once the mechanism makes a mistake, i. e. if it fails to fire 
at the right parity, or if it fires at the wrong parity, that error will he 
remembered, i. e. the parity is now lastingly altered, until there occurs 
a new mistake. A single mistake thus destroys the memory of this par¬ 
ticular machine for all earlier events. In multiplex systems, single 
errors are not necessarily disastrous: But without the "random” permu¬ 
tations introduced in section 9, accumulated mistakes can be still danger 
ous. 

To be more specific; Consider the network shown in Figure 35 

but without the line-permuting "black box" 1£ „ If each output line is now 

fed into its input line (i. e. into the one with the same number from 

above), then pulses return to the identical organ from which they started, 
and so the whole organ is in fact a sum of separate organs according to 

Figure 39, and hence it is just as subject to error as a single one of 

those organs acting independently. However, if a permutation of the 

bllTlHlp "4 C uifA-r-noc ^.A S. w .9 • __ ■*-* * *«, , * 

xds 5Uuwu ? tai Qy ut# m r Igxifre then 

the accuracy of the system may be (statistically) improved. This is, of 

course, the trait which is being looked for by tire insertion of It, i. e. 
of a "random" permutation in the sense cf section 9, But how is it pos- ' 
sible to perform a "random" permutation? „ ' 1,: " ' v 

The problem is not immediately rigorously defined. It is, 
however, quite proper to reinterpret it as. a problem that^caa be stated .. 
in a rigorous form, namely: It. is desired to find one or more permuta-* 
lions which can fee used in the "black boxes’’..marked'with in, or Li {s li.,, ’ 
in the relevant Figure's 35, 37,..so that the essential statistical proper-* 
ties that are-asserted there are truly present. . ‘Let us .consider'the 
simpler one of these two, i. e, the multiplexed version of the simple 
memory organ of Figure 39 -- i. s, a specific embodiment of Figure 35. 

). he discussion given in 10. 3 shows that it is desirable, that the per - • ’ 
mutation Tt of Figure 35 permute the even lines among each other, and 
the odd lines among each other. A possible rigorous variant of the 
question that should now be asked is this. ' ' 

Find a fiduciary Level a >0 and a probability g >Q , such 
that for any e q>Q and any b~ there exists an’ f\fm a) 

and a permutation oL - {Jj^ \ satisfying the following requirement: 

Assume that the'probability of error in a single operation of any given 
Scheffer organ is £ . Assume that at the time J c all lines of the above 
network are stimulated, or that all are not stimulated. Then the num¬ 
ber of lines stimulated at the time ■& + & will be >■ (i-ix/tf or 3$ £ijf, 

respectively, with a probability % / -£ . In addition JTCjj, «)'■*- C 2niS/n) 
s with a constant Q (which should not fes tsxcsssivsly ?rs*t) a 

Note, that the results of section 10 make the surmise seem 
plausible, that & s . 07, £ » ,005 and C lo t OOO/C Iq J ^ 

'vSOO srs suitable choices for the above purpose. 

, OYj The following surmise concerning the a^ure of the per- 
has a certain plausibility; Let AT “ '■ *"--‘ J -~ 


mutation 


the Of complexes (d,,d Zj ^ dg ) (^ s 0 } i for A ■ . 

Let these correspond in some one to one way to the 2? integers 


Consider 


)■ 


1 


M 


( 28 ) - i 

Now let the mapping 
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be induced, under the correspondence (28), by fche mapping 
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fte V c“re y spona e V n a cW2al° £ r ?°" “ ‘dependent of the choice of 

ne correspondence (28), Now (30) cces not change the parity of 

not change the par’^ - *• «• (29), should 

rejon^noe *28*^0 cioi^h. ^‘‘rh rta1 ?^ ' UW11 ' d ’ ««• c «- 

T f , ' chosen as oo let. i nave the same parity as 

A “ * Th , 1S i f^ l , earl y Possible, since on either side each 


parity occurs precise 
above requirements 
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il. 3 Remarks concerning the simplified probability assumption. 

' t he tblrd rern srk on multiplexed automata concerns the as 

This a' un*“li *i'c of -, time and »? »“ Pilous inpots, 

feiiuee fox tSe SehSL" oxg?n oTri f 

£ S £ ss &. 3 ~« 3 £ 

assumpHoSsT"- he Rrobtem’SiVSw lorPlrtUttP™ 

finding the intersection of®°, f f “lT i<Jn * i «•»"* 
allowed) end finally, carrying out the sijisticj ' ™ V s 
more complicated situation. This wilt not be aJtempSd here.' 

12 ANALOG POSSIBILITIES. 


12. i 


Further remarks concerning analog procedures, 


h, c b »e- de-elo-^ - ^ reaS + ° n f ° r thinkin 8 that tiie system which 
any exiting^ **'*>«' »«W<* 
machine or logical machine T _ rf a ‘‘ R. ^■ n '" ia error-safe computing 

due largely to the influence of ihe^echnA f °<fT ° f . OU3, system is 

puting and to fche trends of the d ® vel0 Ped for digital com- 

Now, speaking specifically of the human nervous sysUm^this’‘i S ‘ 

which 1 we S are famUiar ” ^ l “«** ^“nefact with 

and complex. Its duties innuJ: ZL * il\‘ 1 !!_ & ” i f orr ® s P 0nd i n 8 l y varied 
stimuli, of reports of nhv«i«»r °t external sensory 

motor activities *a nd° olP inteTrua 1 ^~he mi cal. 1 le ve* s^th ° n S ’ ** « 

with its very complicated procedure for ? tl? me ™° ry function 

search for information, and of cou^e r ^nsiormatioti of and the 
orders and of more or'ies* aua'-°ufcmuGus relaying of coded 
handle *u +>,.«. -ess quantitative messages. It is possible to 

nuncue ail these processes by dtn-ita? m<.<■!, ^a~ P , . 1Qie to 

and .hen, in .he ^ ”Sf SSS, 
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coding tricks, in the decimal or some other system), and to process 
the digitalized, and usually numericized, information by algebraical 
(i, e, basically arithmetical) methods. This is probably the way, how 
a human designer would at prestsnt approach such a problem. It was 
pointed out in tha discussion in 11.1, that the available evidence, 
though scanty and inadequate, rather tends to indicate that the human 
nervous system uses different principles and procedures. Thus mes¬ 
sage pulse trains seem to convey meaning by certain analogic traits 
(within the pulse notation -- i. e. this seems to be a mixed, part digital, 
part analog system), Like the time density of pulses in one line, cor¬ 
relations of the pulse time series between different lines in a bundle, 
etc. 

Hence our multiplexed system might come to resemble the 
basic traits of the human nervous system more closely, if we atten¬ 
uated its rigidly discrete and digital character in some respects. The 

simplest step in this direfction, which is rather directly suggested by the 

above remarks about the human nervous system, would seem to be this, 

12. 2 A possible analog procedure, 

12. 2. i The set up. 

In su r prototype network $i. each line carries a "yes" (i. e. 
stimulation) or a "no" (i. e, non-stimulation) message -- these are 
interpreted as digits I and O , respectively. Correspondingly, in 
the final/multiplexed) network (which is derived from it ) each 

bundle carries a "yes" s I (1, e, prevalent stimulation) or & "no" 3 
20 (i. e.- prevalent non-stimulation) message. Thus.only two 

meaningful states, L e. average levels of excitation f , are allowed 
for a bundle -- actually for one of these |-v 1 and for the other |^0 . 

Now for large bundle sizes if the average excitation levei$is an 
approximately continuous quantity (in the interval $■& f a. 1 ) -- the 
larger Ji the better the approximation. It is therefore not unreason¬ 

able to try to evolve a system in which | is treated, as a continuous 
quantity in Ss |S I . This means an analog procedure (or rather, 
in the sense discussed above, a mixed, part digital, part analog pro¬ 
cedure). The possibility of developing such a system depends, of 
course, on finding suitable algebraic procedures that fit into it, and 
being able to assure its stability in the mathematical sense (he. ade¬ 
quate precision) and in the logical sense (i, e. adequate control of 
errors). To this subject we will now devote a few remarks. 

12,2,2 The ryp. 110*0- S * 

Consider a multiplex automaton o£ the type which has just 
been considered in 12. 2. 1, with bundle size N . Let § denote the 
level of excitation of the bundle at any point, that is, the relative 
number of excited lines. With this interpretation, the automaton is 
a mechanism which performs certain numerical operations on a set 
of numbers to give a new number (or numbers). This method of 
interpreting a computer has some advantages, as well as some dis¬ 
advantages in comparison with the digital, "all or nothing”, inter¬ 
pretation, The conspicuous advantage is that such an interpretation 
allows the machine to carry more information with fewer components 
than a corresponding digital automaton. A second advantage is that 
it is very easy to construct an automaton which will perform the 
elementary operations ot arithmetics. fOr, to be more precise: 

An adequate subset of 
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these. Cf. the discussion in 12, 3) For example, given § and ■» , it is pos¬ 
sible to obtain k ( l-s-'vj } as shown in Figure 40. Similarly, it is possible to 

T* W 4_ f : _ w \ V. f c-v -«* , .-,2 &."L. - /n,£_".... _ . * » * * 

~ * *-* ^ wifciA u<oftf |, iui cuura€, mere musi * 

be o{ = H/Nj H = 0,1.-, ^ 


w }. bli ij « 

but this ranee for «; 


continuum." as that one for £ , hence w 


is the same "approximate 


properly as the latter. 


may treat the former as a continuum 

!«/„ -need only choose c<J\f lines from the first 


le them with 0-=«>JV lines from the second. To obtain the 


just 

bundle and combss 

quantity 1- |yj requires the following set-up shown in Figure 41. Finally we 
can produce any constant excitation level & {O^ i J ( hy originating a bundle 

so that ocJV lines come from a live source and Ct-*1 jsT from ground. 

12. 3 Discussion of tbs algebraical calculus resulting from the above 
operations. 

Thus our present analog system can be used to build up a system of 
algebra where the fundamental operations are 


Oi 


/•ai \ 


Oi *| 4- C i — 0i ) Y) 


j {for any constant at in 

] O'* <x * O 


V 

i > 


1 


All these are to be viewed as functions of f, ^ , They lead to a system, in 
which one cjm-operate freely with all those functions £Cf, £*..f t< ) of 
any k variables f1- A , .,. , that the functions of (31) generate/ I. e. with 

all functions that can be obtained by any succession of the following proces- 

s e s: 


(A) In the functions of (31) replace <n by any variables f, 

(B) In a function y 


. =1 f 5 JTf 

replace the variables Si „ 

‘ j c ,, ; 

obtained. 


, that has already been obtained, 
cf , by any functions g, ( f Sj ., | t4 f 
; respectively, that have already been 


xo tnese, purely algebraical-combinatorial processes we add a properly 
analytical one, which seems justified, since we have been dealing with ap¬ 
proximative procedures, anyway: 


(C) If a sequence of functions i C j,. 


Ik) , 


u = \ 


£ h/ 



,*S 


.s: 


ttf-j 

O 

* 
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that have already been obtained; converges uniformly (in the 
domain Dtf % x * 5 , .«• ^ 0<s f | \ for 


to fctv 


then form this ■? ( l s 


Note, that in order to have the freedom of operation as express¬ 
ed by (A), (B), the same ‘'randomness" conditions must be postulated as in 

the corresponding parts o£ sections 9 and 10. Hence "randomizing" permut¬ 
ations T£ must be interposed between consecutive executive organs (i. e. 


ferred to above. 


ftuwv^ emu. au 11g ju.a l- clzs m tae ettscCluuS 


In ordinary algebra the basic functions are different ones, namely: 

( o( | (for any constant in 

S. . \ O £ 9( dS i ) . 


It is easily seen, that the system (31) can be generated (by 
(A), (B)) from the system (32), while the reverse is not obvious (not even 
with (C) added). In fact (31) is intrinsically more special than (32), i, e. 
the functions that (31) generates are fewer than those that (32) generates 
(this is true for (A), (B), and also for (A), (B), (C)) -» the former do not 
even include « Indeed all functions of (31), i. e, of (A) based ©a (31), 

have this property: If ail variables Us in the interval Os f is I , then the 
inaction, too, lies in that interval. This property is conserved under the 
applications of (B), (C), On the other hand § «!»^ does not possess this 
property -- hence it cannot be generated by (A), ,(B), (C) from (31). (Note, 
that the above property of the functions of (31), and of all those that they 
generate, is a quite natural one: They are all dealing with excitation levels, 
and excitation levels must, by their nature, be numbers f with 0-& f rfe I . ) 
In spite of this limitation, which seems to mark it as essential¬ 
ly narrower than conventional algebra, the system of functions generated(by 
(A), (B), (G); from (31) is broad enough for all reasonable purposes. In¬ 
deed, it can be shown that the functions so generated comprise precisely 
the following class of functions: 

All functions j'C 3 %.^... f fe ) which, as long as their vari¬ 
ables 'fa j j § K _ lie in the interval 0-& §■& l, are continuous and have their 
value lying in that interval, too. 

We will not give the proof here, it runs along quite conven¬ 
tional lines. 


12.4 Limitations of this system. 

This result makes it clear, that the above analog system, i. e. 
the system of (31), guarantees for numbers f with 0& f # f (i. e. for the 
numbers that it deals with, namely excitation levels) the full freedom of 

algebra and of analysis. 

In view of these facts, this analog system, would seem to have 
clear superiority over the digital one.. Unfortunately, the difficulty of main¬ 
taining accuracy levels counterbalances the advantages to a large extent. 



i ite 
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'Stl 10 SCj£ 


ijL v vU« y 0 « 5 s,ii iiv v %!4 0 { ~ Jp 1 0 '■* 

is an intrinsic noise level of the order </A , i. e., 

e * if, r - *— *• - «««’■ 

i?« & ©uavL ^ we . 


a £n -o^iissr words* there 

for the Jv considered in 

10. 5. 2 and 10, 5. 3 {up to <v2ft, 000) at best iCT*. Moreover, in its effects on 

the operations oC (51), this noise level rises Cross, w to . (E. g. * 

for the operation 1*1^ * cf, 'the result (21) and the argument that ie&ds to it*) 

With the above assumptions, this is at best ^ 2 0“2, %,s. 1% f. Hence after a 
moderate number of one rations, the excitation levels are more likely to re- 

It should he emphasised, however* that this is not a conclusive 
argument that the human nervous system doss not utilise the analog system* 

As was pointed out earlier, it is in fact known for at least some nervous pro¬ 
cesses that they are of an analog nature* and that the explanation of this may, 
at least in part, lie in the feet that the “logical depth” of the nervous network 

is quite shallow in some relevant places. To be more specific: The number 

of synapses of neurons from the peripheral sensory organs, down the affer¬ 
ent nerve fibres# ferough the : brain, bask through the efferent nerves to the 


os course tne p; 

j* - ?«. * __, 

{■ '•«* 0 4^»li u Hi 


yauei cempses- 


:or system may act be mors than ^i.8* 
ity cf tbs network of neurons ia indisputable* * 

back in the human brain may be overcome by some kind of .self-stabilisation. 

At the same Sase, a good argument can be put up that the animal nervous 


odueed by feed* 


system uses analog methods (as they are iat 
est way, accuracy being a very minor consideration* 


«efee4 above) only 


w iW 1 * 


in UtB C3*U.52> 


12* 5 A plausible analog mechanisms Density modulation by fatigue. 


Two more remarks should be made at this point. The first ©as 
deals with some more specific aspects of fee analog element ia the organisa¬ 
tion and functioning of the human nervous system. The second relates to the 
possibility ©? stabilising the precision level of the analog procedure that was 
outlined above. 

This is the first remark. As we have mentioned earlier, many 

neurons of the nervous system transmit intensities (i. e. quantitative data) 

by analog methods#' but# in a way entirely different from fee method describ- 


__ i -> *» 

Lti 4 . £** £*$ 




«**»» *•* / ^ ao w ® 

’•he single nerve 


of a bundle of nerve fibres) varying, as described in 12* 2, 

fibres tire repetitiousiy, but with varying frequency ia time. For example, 
the nerves transmitting a pressure stimulus may vary in frequency between, 
say* 6 firings per second and* say, 60 firing® per second. This frequency 
is a monotone function of the pressure* Another example is the optic nerve, 
where a certain aei oi fibres responds in a similar manner to the intensity 
of the incoming light. This kind of behavior, is explained by the mechanism 
of neuron operation, and in particular wife the phenomena of threshold and 
of fatigue. Wife any peripheral neuron at any time can he associated a 
'threshold intensity: A stimulus will make fee neuron fire if and only if its 
magnitude exceeds the threshold intensity. The behavior of the threshold 
intensity as a function of the time after a typical neuron fires is qualitatively 
pictured in. -Figure 42* After firing* -there is an "absolute refractory period" 
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of about 5 milliseconds* during which no stimulus can ma*e the neuron 
fire again. During this period, the threshold vame is inamte, 
comes a "relative refractory period." of about 10 milliseconds, aurmg 

which time the threshold level drops back to its equilibr^ v^lue may, 
even oscillate about this value a few times at the end). This decrease 

, , _ j. __■kt.-.w, +'h a new will fire again as soon 

is for the most part 4 uuuuwn»t. •»* --- -- 

as it is stimulated with an intensity greater than us excitation thres* 
hold Thus if the neuron is subjected to continual excitation of constant 
intensity (above the equilibrium intensity), it will fire periodically with 
a period between 5 and 15 milliseconds, depending on the intensity of 

the stimulus. , , . , , 

Another interesting example of a nerve network winch trans¬ 
mits intensity by this means is the hurhan acoustic system. The ear 

analyzes a sound wave into its component frequencies. These are trams- 
mitted to the brain through different nerve fibres with the intensity 
variations of the corresponding component represented by the tre- 

Quency modulation of nerve firing. 

The chief purpose of all this discussion of nervous systems 
is to point up the fact that it is dangerous to identify the real physical 
(or biological) world with the models which are constructed to explain 

it. The problem of understanding the animal nervous action is far 
deeper than the problem of understanding the mechanism of a comput¬ 
ing machine, Even plausible explanations of nervous reaction saouiu 
be taken with a very large grain of salt. 

12, 6 Stabilization of the analog system. 

We now' come to the second remark. It was pointed out 
earlier, that the analog mechanism that we discussed may have a way o 
stabilizing excitation levels to a certain precision for its computing 

operations. This can be done in the following way, 

For the digital computer, the problem was to stabilise me 
excitation level at (or near) the two values © and 5 . This was accom¬ 
plished by repeatedly passing the bundle through a simple mechanism 
which changed an excitation level % into the leve. V- 1 !! • ^ !, ' re 

function f C%') had the general form shown its Figure 4s. The reason 
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that such a mechanism is a restoring organ for the excitation Levels f v 0 
and | «n! i (us, that it stabilises at *» or near — 0 and 1 ) is that J tf) 
has this property? For some suitable &• (Os trst) implies 

0<fCf)<^ ; &-<!<! implies I < $Cf) < \ . Thus fa 0 S I are the only 

stable fixpcints of $(%) . (C£, the discussion in 9.2,3 and 9, 4. 2 ,) 

" ~-»**~*^*«-* y'** * vVuACn aas the Korra shown is 

Figure 44, I, e, we have; 

0 s Ctg ^ ^ 4 0 5 ., C , i. 4 4 ^ ^ I ^ 


for iah 


a.-. s < |< I implies 
h, <\< a*, implies 


H-i <tf $ i%\ •*£ | 

4 «4 4C| j« at 


Here S,(*0 } } O|j,m ^ &y. kJ &*>£#!) are *££§) 4 only stable fixpointa, 
and such a mechanism is a restoring organ for the excitation levels 

f„ ~ ©feCafl) s a,,,., ,Oa.i J a* O . Choose, e. g, &i 8 i/p Ct» 0, 
withy <6 , or more generally, just fi.; «* liae l 9 ,„ f p) with some 

suitable v , Ihen tnis restoring organ clearly conserves precisions of 

the order 6 (with the same prevalent probability with whtth it restores). 

13 CONCLUDING REMARK. 


13, 1 A possible neurological interpretation. 

There remains the question, whether such a mechanism is possi¬ 
ble. with the means that we are now envisaging. We have seen further above, 
that this is the case, if a function yf fj with the properties just described 
can be generated from (31), Such a function can. indeed be so generated. 
Indeed, this follows immediately from the gesieral characterization of the 
class of functions that can be generated from (31), discussed in 12, 3. How¬ 
ever, we will not go here into thi3 matter any fur the r. 

It is not inconceivable that some M neural pools" in the human nervous 
system may be such restoring organs, to maintain accuracy in those parts 
of the network where the analog principle is used, and where there is” 
enough logical depta" (ct. ;E. 4) to make this type of siabiliisation necessary. 
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